Changing hostname in postfix config

Setting a proper hostname for mail servers is a small step, but often required, especially you’re looking to do everything possible to avoid spam ratings in your email.

Having recently completed a fresh install of postfix on a new mail server, I proceeded with running a test email through Spam Assassin to see if any adjustments were needed.

Before I go on … if you’re applications send email and you’re not using a tool like Spam Assassin to gauge spam-potential of your emails, I highly recommend that you start.  Our apps send everything from account-creation emails to invoices, so it’s required that our product emails are trusted, sent, and received without issue.  Spam Assassin has always helped us achieve that.

In this case, Spam Assassin reported a HELO_LOCALHOST flag.  A test against our mail server’s hostname revealed that it was still identifying itself as localhost, which is a potential spam indicator.

Fortunately resolving this is easy, we just needed to change our mail server’s hostname. That’s easy to do with postfix, it just requires 2 steps.

First, change the myhostname directive in /etc/postfix/main.cf to the desired hostname.  In our case we changed it to timepanel.net (the server name for our time tracking application, TimePanel):

# myhostname = localhost
myhostname = timepanel.net

Second, restart postfix:

sudo service postfix restart

And that’s it!  After this change Spam Assassin no longer reports a failed test on our mail server’s hostname.

Posted in General, Linux | Leave a comment

Google Chrome Stuck In Full Screen Fix

If you’re a Google Chrome user you may have encountered a bug where the browser window gets stuck in full screen mode.  In such cases, hitting F-11, or clicking the Exit Full Screen link does nothing.

After searching for fixes, many of which were ineffective, the only fix I found to work was deleting my user preference directory.  If you’re a Linux user, here’s how:

First, go to the google-chrome directory in your home path:

 cd ~/.config/google-chrome

Next, verify you have a Default directory.  Rename it as a backup resource, just in case you need it.

 mv Default/ Default_backup/

Then restart Chrome.  You should see that a new Default directory is created.  You should also notice Chrome starts normally and is no longer stuck in full screen.  The only drawback here is that your preferences are trashed, but most will find that more than acceptable since Chrome is now usable again.

Posted in General | 6 Comments

Merging 2 WordPress Blogs Is Easy

Several months back we made the decision to merge TimePanel’s blog into our company’s blog here at solutionfactor.net.  Fortunately it was a breeze thanks to native support within WordPress to do so. Here’s how we did it; hopefully this saves readers some time if they’re looking to do something similar.

Step 1:  Backups

Backups are always a good idea, especially in cases like this where data will go through changes that you may want to revert, either due to unexpected error, or just b/c you change your mind at some point.

You can backup both site’s files easily using cp to copy files to a backup directory:

sudo cp -r blog_1_directory/ blog_1_backup/
sudo cp -r blog_2_directory/ blog_2_backup/

 … and for the databases, a vanilla mysqldump handles the job:

sudo mysqldump -u -p blog_1_database > blog_1_database.sql
sudo mysqldump -u -p blog_2_database > blog_2_database.sql

Step 2:  Export The Source Blog

Next up is exporting the source blog.  By source blog, I mean the blog you’re exporting data from.  By contrast, I’ll refer to the target blog (later) as the blog you want to import data into.

Log into the source blog and go to Tools > Export. From there, select to export All content (it’s probably selected by default), then click Download Export File.

WordPress Export Blog Page

WordPress Export Blog Page

Step 3:  Import Into The Target Blog

With the source data exported, next up is to import the data into the target blog.  To do so, log into the target blog and go to Settings > Import.

From there, you’ll see an option to install an import tool; select the WordPress tool at the bottom of the list.

Once installed, run it.   You’ll see an option to create a new author for the imported posts, or assign to an existing author. In our case, we chose the latter, but choose the option that best suits you.

And that should be it.  Once the import tool completes, you should see all your blog posts on the new blog.

Step 4:  301 Redirects

If you’re planning on removing the old blog posts once they’ve been imported into the target site, you’ll want to add 301 redirects to inform search engines the content has moved.  Otherwise, the old content will produce 404 errors once it’s gone, which can lead to SEO penalties.

Fortunately it’s an easy change to make.  You can add the following redirect to your vhost or .htaccess file:

redirect 301 [old blog url] [new blog url]

In our case, that translated to the following:

redirect 301 /blog http://solutionfactor.net/blog/

Posted in General | Leave a comment

Using Siege and KCacheGrind to Optimize SolutionPHP’s ORM

Some time ago we posted about how to install XDebug and KCacheGrind.  I think anyone serious about PHP performance should keep those tools handy and in regular use.

And it just so happens they helped us uncover a huge performance bug.

We recently started developing our own MVC framework (SolutionPHP) for use on our company projects.  One of our decisions was to include an ORM implementation.  Even though we’re generally not fans of ORMs, we’ll easily agree that there are times when it makes sense to use them.  Adding the base orm functionality has been pretty straightforward and a nice learning experience.

After our first pass at SolutionPHP’s ORM, we wanted to gauge its performance trade offs.  To do so, we benchmarked 2 branches of a site’s home page.  Both branches produce identical page markup and fetch the same data from the database, the only difference being that one uses the ORM and one does not.

To start, here’s SolutionPHP’s performance baseline of the sites’s home page:

SolutionPHP's performance baseline: 468 requests/sec.

SolutionPHP’s performance baseline: 468 requests/sec.

This is w/out the user of the ORM; we see about 468 requests/sec.  This is very fast for an MVC framework that’s hitting a database!  You can view performance for other frameworks here.  Note those benchmarks are on minimal Hello World sites that don’t touch the database.  We immediately see that SolutionPHP’s performance is going to be one of it’s stand out features.

Next up we switched to the ORM branch and ran a baseline:

SolutionPHP performance baseline, with use of its ORM.

SolutionPHP performance baseline, with use of its ORM.

Yikes, a measly 59 requests/sec!  And just like that, the framework’s performance is kaput.  Let’s take a look at where things go wrong.

Here’s a profile screen shot in KCacheGrind:

KCacheGrind showing SolutionPHP's ORM profile, before optimizations.

KCacheGrind showing SolutionPHP’s ORM profile, before optimizations.

The entries worth noting are boxed in red.  There looks to be 30-31 recursive calls on certain methods.  Sounds like the typical N + 1 query problem ORMs are notorious for, and one of the primary reasons why they cause performance problems.

And brief investigation confirmed that.  Those recursive calls are firing once for each row found in a result set.  Of those methods, the one I was concerned with was list_columns(), the second call from the top.

list_columns() is responsible for reading a table’s columns so that the ORM knows which properties to attach to a model object.  For example, assume a user model based on a user database table that has id, first_name, and last_name columns.  list_columns() would query the database for the user table’s columns and inform the ORM that a user model object should have id, first_name, and last_name properties attached to it.  Its code and database query are simple enough:

public function list_columns($table_name)
{
    return $this->fetch_all('SHOW FULL COLUMNS FROM '.$table_name);
}

Back to why this causes an N + 1 query problem …

Such logic makes sense, at least as explained above – models knowing which attributes it needs to be aware of is a good thing (although there are other ways to accomplish the same thing, such as hard coding and maintaining a list of attributes directly in each model, perhaps via a private or protected array property).  But! … taking a step back, it makes sense that a model shouldn’t have to be reinformed of its attributes multiple times for a given result; certainly not once per row, especially if doing so requires a database query each time!  It should only happen once.

And the fix was simple enough.  Instead of calling list_columns() N times, we made two revisions:

  1. Models now call list_columns() in their constructor, then cache the result in a local member variable, and
  2. The ORM base class now references that member variable when mapping database columns to model attributes – no more calling list_columns() multiple times for this!

The difference in performance is rather incredible.  Running the same benchmark now brings us up to 380 requests/sec!

Requests/sec after the ORM optimization.

Requests/sec after the ORM optimization.

Conclusion

Making simple performance mistakes is easy, especially when you’re working with something like an ORM that interfaces with a database.  However, tools like XDebug and KCacheGrind can provide visibility into those problems in a way that makes them readily apparent and therefore easier to address.

Posted in Database, General, Performance, PHP | Leave a comment

Ensure Correct Content-type for XHR2 FormData

Just a heads up that if you’re submitting data via FormData with XHR2, you’ll need to ensure the proper Content-type header is set, otherwise request data won’t be sent properly. In my case, an incorrect Content-type header resulted in only sending FormData’s first attribute with the request.

Here’s some detail.

This was the form in question; Signal’s login form code, note the email and password fields:

Login form markup

Login form markup

Straightforward enough. But here’s what an XHR2 request looked like in Chrome’s Dev Tools:

XHR2 request, only sending the form's first element value.

XHR2 request, only sending the form’s first element value.

Note that only the email value is being sent. Something is off.

Turns out the Content-type request header was set incorrectly. As shown above, it was set to application-x-www-form-urlencoded. Obviously that Content-type is incorrect given that we’re no longer sending request data via an encoded url, but rather as form data.

I had left the incorrect urlencoded header value in by acccident, but after removing it, all values from the login form are now sent as expected:

XHR2 after Content-type request header fix.

XHR2 after Content-type request header fix.

Hope this saves someone some time.

Posted in General | Leave a comment

Adding PHPUnit To LAMP Projects

Since we had a client ask about the effort involved to setup automated testing in their product, and because I mentioned it before in a LAMP Best Practices post, we thought it’d be a great idea to share how easy it is to get PHPUnit going in a new project.

Prerequistes

Our unit testing framework of choice is PHPUnit.  So before we get started, we’ll need to get that installed.  At Solution Factor we all develop on Linux Mint, so installing it from the repo is simple (note: the following commands only apply to Debian based distributions).

To install PHPUnit, run:
[code lang=”bash”]sudo pear install phpunit/PHPUnit[/code]

Next, a CodeCoverage library may be needed; unfortunately it’s left out of some PHPUnit packages.  To install it, we just ran the following:

sudo pear channel-discover pear.symfony-project.com
sudo pear channel-discover pear.phpunit.de
sudo pear channel-discover components.ez.no
sudo pear install phpunit/PHP_CodeCoverage

With that, our PHPUnit install is complete. Now we’re ready to start adding PHPUnit to our project.

Directory And File Setup

To get PHPUnit setup, we started by creating a tests directory.  This can be created anywhere in a project, so long as it’s outside the document root (for security purposes).   The tests directory will house everything test related, and within it we created 3 things:

  1. a phpunit.xml file – the config file for phpunit
  2. classes/baseTest.php – will house a simple test to confirm PHPUnit is working
  3. bootstrap.php – to allow PHPUnit to bootstrap into our project

Here’s what the basic file and directory setup looks like in my dev environment:

PHPUnit file structure

PHPUnit file structure

PHPUnit File Modifications

With the file and directory structure setup, it’s time to inform PHPUnit about our test suite, and provide it with tests to run.

First up is the phpunit.xml file.   This is PHPUnit’s config file and tells it where tests and any bootstrap files are. Here’s the content we added, note the Bootstrap and test suite settings:

<!--?xml version="1.0" encoding="UTF-8"?-->

./

Next we added a simple test case to classes/baseTest.php file.  The test isn’t meaningful, it only serves as a way to confirm PHPUnit is running properly.  Here’s the file content:

<?php

class BaseTest extends PHPUnit_Framework_TestCase
{
    public function testBase()
    {
        $this->assertTrue(true);
        $this->assertEquals(1, 1);
    }
}

And that completes the setup. To confirm PHPUnit runs fine, cd to the tests directory and run phpunit.  We initially got a bunch of HTML output, and you may too, but at the very bottom PHPUnit did give the following confirmation:  OK (1 test, 2 assertions).

So if you see that, phpunit is working.

The Bootstrap File, And Getting Rid Of HTML Output

We didn’t mention the Bootstrap file much, but we did just mention some unneeded HTML output.  Both are related.

The Bootstrap file for PHPUnit serves a similar purpose as it would for other frameworks (i.e. most common MVC frameworks):  it helps PHPUnit initialize and load any config or libraries needed to run tests properly.  For example, you should aim to provide PHPUnit with the same config parameters that your app has, for things like database connections, path information, etc. An example of how we utilized a bootstrap.php file with teh Kohana framework can be found here: PHPUnit into Kohana.

Fortunately in our case little was needed in bootstrap.php. All we added was one line of code to load the config file for the app we’re testing:

<?php
require('../app/config/config.php');
?>

With that, PHPUnit then had access to the same config parameters our app does. Without loading the app’s config in bootstrap.php, we’d have to redefine the same config values somehow for PHPUnit; the bootstrap.php file allows you to avoid having to do that in a redundant way.

For your app, expect that bootstrap.php will serve a similar purpose, but obviously will contain different code than what we added.

Conclusion

We believe every LAMP project should integrate automated testing. As shown here, it’s easy to get that started. Once setup, reaping the benefits of automated testing is only a matter of adding tests as the code base progresses. And, once matured, the test suite is a great way to gain confidence that new features, bug fixes, or other changes are stable as they move upstream to production.

Posted in General | 4 Comments

Preload and Async Scripts For Faster Load Times

Keeping our products fast will always be one of our primary goals.  We love fast apps, and keep a lookout for ways to make them even faster.  That goes for initial load times as well as usage while inside the app.

Unfortunately JavaScript heavy apps fight against that goal.  For example, as additional features make their way into TimePanel, the amount of JavaScript we have to send across the network increases.  Caching helps, but has its own limits.  For example, the first time a user logs into TimePanel, they’re served the app’s CSS and JavaScript due to a cold cache.  Additionally, anytime we deploy updates, TimePanel’s CSS and JavaScript version numbers are bumped, forcing new versions to be loaded when a user logs in.  Those are two scenarios where caching doesn’t help and load times are increased.

Normally those kinds of circumstances may not be that bad, but since TimePanel’s JavaScript contain the entire app, its size introduces extra load time that users have to sit through.  That JavaScripts lock the browser’s UI thread until they’re downloaded and parsed compounds the matter.

Due to those problems, and how much we value speed, we sought out a way to minimize load times.  Enter HTML 5’s async script attribute coupled with a simple preloading technique.

JavaScript, HTML 5 Async, and Preloading

To give a brief background, there’s one main reason why JavaScripts, and their size, are problematic.  They block the browser’s UI thread, meaning that until a JavaScript is downloaded and parsed, nothing else happens in the browser.  That means a user could see nothing happening for some moment of time, perhaps even seconds, if a given JavaScript is big enough, or network latency is high enough.  This is why it’s always a good idea to keep JavaScript small, and if possible, serve them from a fast CDN (a CDN will typically provide better geolocation, speeding up file delivery to the user).

Another way to combat that is to leverage HTML 5’s async script attribute, and preloading.

HTML 5 introduced the async script attribute to mitigate the negative affects we just described related to JavaScript files.  With the async script attribute, developers have a way to declare that a script be loaded asynchronously so as to not block UI rendering or subsequent file downloads.  This provides a huge step forward toward maintaining a fast user experience while JavaScript files download to the user’s browser.

Additionally, preloading scripts is another way to achieve that.  By selectively inserting script tags in a non-interfering way before they’re needed, a user can transparently download and parse a JavaScript file, or other asset, behind the scenes.  This doesn’t speed up file loading per se, rather it off loads it in a way that’s transparent for the user.  This provides a perceived speed increase, and is exactly the kind of change we just deployed to help TimePanel load faster.

The Changes, and Before/After Results

The opportunity we saw was to load TimePanel’s CSS and JavaScript on the login page.  Since we didn’t want to just move the script load time from post-login to pre-login, we added the async script attribute to the CSS and JavaScript script tags.

Edit:  Thanks to Silvenga for correcting my use of the async attribute for CSS.  He correctly noted that async attributes only apply to script nodes, not link nodes.  I’ve since edited my post above to strike-through CSS references, and updated the following screen shot.

Async attribute.

Async attribute added to JavaScript file.

That keeps the login page rendering practically unaffected.  The JavaScript file downloads asynchronously while the user views the login page, or while entering their credentials.  Then, after the user logs in those files are reloaded, but this time via browser cache, resulting in a decrease in TimePanel’s load time, post-login.

Again, the download times of TimePanel’s JS using async on the login page isn’t faster than if it were downloaded otherwise.  But, since we’re doing it behind the scenes while the user is logging in it increases perceived performance of the app after login by being able to serve TimePanel’s JS from the browser cache instead of over the network.

And the results are noticeable.  Before and after results, taken from Chrome’s network waterfall, show the onload time went from 926ms to 646ms, a decrease of 283ms!

Before async and preloading - 926ms onload.

Before async and preloading – 926ms onload.

 

After async and preloading - 646ms onload.

After async and preloading – 646ms onload.

Conclusion

Speed is important, and HTML 5 and other modern techniques offer opportunities to speed up modern applications, despite growing feature sets and file sizes.

If JavaScript, or other performance-sensitive assets, are getting in the way of load times, consider applying the async script attribute to keep the user’s browser UI from locking up.  Also consider preloading scripts when possible to have scripts loaded and at the ready by the time user’s need them, without making the script’s download and parse time noticeable to the user.

 

Posted in General | 4 Comments

7 Best Practices For LAMP Projects

As a consultancy we’ve encountered all kinds of code.  Often times we inherit code from other teams, in which case we always make it a priority to review its style, structure, how it’s been deployed, if any frameworks were used, etc. Being able to understand and pick up another team’s work is often part of the job.

However, there’s no denying that sometimes things require extra discovery and adjustment to help course correct certain parts of a code base. n light of that, and having discussed this same topic a few times in the past month, we’d like to share 7 best practices we rely on to keep our client’s products, and our own, trouble free and enjoyable to develop.

Start & Stay With Up-To-Date Versions of LAMP

Running an outdated tech stack can result in costly bugs, security issues, or performance problems. For new projects especially, or legacy products that will undergo a full port, using an up-to-date LAMP stack helps start things off on the right foot.

One reason is that performance improvements are mostly gratis. For example, PHP 5.2 vs. 5.3 performance improvements are well documented; the same goes for MySQL 5.5.  In addition to what’s documented on the web, simple LAMP upgrades have helped several of our clients fix costly performance problems.  This is one of them.  Even in the absence of performance problems, an up-to-date stack helps ensures products stay snappy and responsive.

Fully Enable Error Reporting

One of PHP’s warts is that it has, at least in the past, shipped with supressed error reporting. This has to be one of the biggest time sinks to the community – all too often forum or community posts can be found where a PHP developer is seeking help with debugging some undesirable behavior, only to have no error output to go on.

The fix for that is to set error_reporting to E_ALL in php.ini.  For non production environments you may also want to turn display_errors on, although you’ll want to leave this off in production for security reasons in favor of only relying on error logs to capture error output.

With error_reporting set to E_ALL, PHP’s error reporting works with you by providing more detailed error output vs. against you by hiding it. With that, you’ll experience much faster debugging and a more enjoyable development experience because of it. You’ll even see things like deprecation warnings to give you advance notice of features or syntax that are slated for removal from the language – having more time to address these items is always a good thing.

Avoid Developing In Shared Environments

One common problem among PHP dev teams is developing in shared environments. That’s to say that things like a central database server or PHP installation are shared among multiple developers.

The problem with shared development environments is they’re a single point of failure for your team’s ability to work. Things like a network outage, or a developer making an erroneous server change, or a hardware failure (just to name a few examples) can all render your team non-productive for hours. Such teams operate with a high risk environment that requires careful and thorough communication when changes are required, and that shouldn’t be the case.

We feel the proper alternative is for each developer to run their own local development environment. By doing so each developer is under almost no risk from the factors mentioned above, and they’re free to experiment and make changes as needed to accomplish their individual project objectives without putting the productivity of others at risk.

One tradeoff to adopting local development environments is that they require more time for each developer to setup and maintain their own environment, vs. if they were to join a shared environment that’s already been setup and perhaps maintained by a team lead. But it’s worth it, arguably, because each team member should have a strong understanding of what it takes to setup a product’s environment, so the additional time to understand all the pieces involved is a wise investment. Things like file structure or location, configuration requirements, etc. all come to the forefront for each developer during their setup vs. remaining unknowns or hidden in the minds of one or two team members. Plus, automating environment setup is absolutely an option, if not preferred, which reduces the time required on the part of each developer to get their environment going.

Version Control

All teams, that use the LAMP stack or not, should be required to operate with some form version control. At the time of writing this Subversion is very popular, but Git, Mercurial, and Perforce are arguably better options. Regardless of what you choose or why, each team deserves a single source of truth when it comes to tracking state for their projects, and version control is probably the only reliable way to accomplish that. Worded another way, if your team isn’t using version control you’re likely doing something wrong.

To illustrate the value one only needs to imagine a production hot-fix scenario. I.e. some bug is found in production and is fixed ASAP. However, a team without version control is at higher risk of making the change willy-nilly and pushing it to the production environment, only later to have it overwritten when the same production environment receives its next update. The culprit is, of course, that the hot-fix was only committed to the production environment and wasn’t properly committed or merged elsewhere.

And so a proper workflow really depends on having version control. An alternative approach to the above would have been to commit the change to a hot-fix branch first, in the product’s version control repository. Even that much provides immediate benefit in that the change exists somewhere that isn’t easily overwritten and can easily be reviewed by all team members (if needed). From the hot-fix branch, sure, it can then be deployed to production, but it can also be easily merged into other branches such as testing, development, feature branches, local environments, etc., thereby removing the risk of being overwritten by an update.

Adopt A Framework

All teams should adopt some framework as a means to adhere to a uniform structure in which to solve a product’s various problems. There’s a wealth of good frameworks out there to choose from (Symphony and Zend Framework are two examples). Good frameworks are even likely to offer a very good foundation for less experienced team members to learn from. Such frameworks will typically provide great documentation resources, handy view templating (if that’s your thing), database abstractions, migration support, etc. But the overall benefit is that good frameworks allow teams to more or less hit the ground running with solving a product’s problems vs. reinventing architecture or conventions that a framework already provides.

Automated Testing

It happens all the time – something changes, something else breaks. It can happen when committing a simple change or when merging entire feature sets. Even worse, sometimes what breaks is so unrelated that it goes unseen or untested by QA because, well, why would something like an invoicing module break when all that’s changed was a fix to a user signup process?

Enter automated testing, probably one of the best “best-practices” of our time.

Any new team or product should really adopt automated testing, the benefits are just too vast. To start, by thinking about what tests should be written in accordance with a given change, developers are really empowered with a deeper understanding of the changes they’re making. Another advantage is that as time goes on, an automated test suite becomes an invaluable investment that pays large dividends toward catching breaking changes before they leave a developer’s environment. The accrual of tests for input validation, or tests that check when-this-changes-so-should-this, or that help ensure a user can not exceed their paid plan’s quota, etc. provide a tight, constant feedback loop as to whether new changes also introduce breaking changes. And the feedback loop is super fast, much faster than a series of commits and pushing changes upstream, where they may or may not be found by a QA team member.

Automated testing does not remove the need for things like QA, but it does provide far more stability and understanding of a product’s changes, especially over time. If you aren’t implementing some form of automated testing, start taking a look at PHP Unit and/or Selenium today.

Automated Deployments

The days of manually deploying web apps should be far in the past. That’s because manual deployments don’t scale, they’re too slow and error prone! Especially in distributed environments or when the number of target servers is high. Often time manual deployments are just flat out impossible.

But even for smaller teams or environments, automated deployments provide a fast, reliable, means to get changes into production (or test, or stage, etc.). Instead of having to handpick which files need to be deployed and then manually applying those to a set of servers, an automated deployment process might instead consist of a shell script or some build technology (ANT, or Maven). And in a simple command, it might access a given server, hook into version control and pull the latest changes, then apply database updates. With the process automated, it can happen very reliably and within seconds, each and every time, vs. a clumsy manual process that errors regularly somewhere along the way.

As mentioned, technologies such as ANT or Maven are popular in the Java Community but can also work for LAMP projects. Jenkins or shell scripts can also be just as effective, depending on the needs of the team or target environments.

Conclusion

LAMP is an extremely popular stack, in part due to its low barrier of entry. But that also means it’s very easy to race past that without considering, or even knowing about, some of the best practices worth considering for your next LAMP project. The start of every project is an opportunity to improve upon the last.  At Solution Factor we believe the best practices above are some of the more beneficial ones to consider, and we hope you do to. They’ve helped increase not only the reliability of our products, but also the fun in developing rock solid applications for our users and clients.

Posted in General, Performance, PHP | Leave a comment

NetBeans is Slow on Ubuntu 10.10 – Switch Your JDK

NetBeans can be slow, very slow.  After trying out everything on the NetBeans Performance FAQ to no avail, I finally reached the point where I decided to switch to another IDE instead of trying to manage NetBeans’ performance problems.  Before doing so, I got a tip from a friend to check my JDK and switch to the Sun JDK if I wasn’t using it already.  Turns out I wasn’t, I was using the OpenJDK all along, so I decided to give the Sun JDK a shot. It actually turned out to be the silver bullet I needed.  After installing it, NetBeans returned to being snappy and responsive again … here’s how to make the change on Ubuntu 10.10.

First, you may need to add the sun-java6 repository before hand, but that’s not that big of a deal. You can run the following 3 commands to add that repository and install the Sun Java 6 JDK:

sudo add-apt-repository ppa:sun-java-community-team/sun-java6
sudo apt-get update
sudo apt-get install sun-java6-jre sun-java6-bin sun-java6-jdk

Once that’s done, you’ll need to update your Java alternatives so that everything that requires the JDK will use the Sun Java 6 JDK vs. whatever other JDK you have installed. To do so, run the following command and choose the Sun JDK:
[code lang=”bash”]sudo update-alternatives –config java[/code]

Once your JDK is updated, you’ll still need to update the JDK that NetBeans uses. To do so, open the netbeans.conf file and update the netbeans_jdkhome directive to point to the new JDK:

Once that’s done restart NetBeans and you’ll be welcomed with a snappy, responsive IDE!

Posted in General, Performance | 7 Comments

MySQL – WordPress & Avoiding unneeded Group By clauses

A client recently reached out in regard to a reproducible performance problem on their WordPress site.  The use case and conditions were as follows:

  • Performance was fine until a change happened to the wp_posts or wp_term_relationships tables (i.e. when a new blog post is created, or an existing post is modified)
  • Once those tables changed, related result sets in the query cache were cleared, rebuilt, and re-cached
  • During re-caching, the site was brought to it’s knees due to (otherwise performant) queries executing against an empty cache.  Once re-caching completed, those queries returned their result sets much faster, and everything returned to normal.

Fortunately, there had already been some investigation by the time I began troubleshooting.  The client had their mysql long_query_time set to 10 seconds, and the slow query log was reporting tens of variants of this query during problematic conditions:

SELECT wp_posts.* FROM wp_posts
INNER JOIN wp_term_relationships ON wp_posts.ID = wp_term_relationships.object_id
WHERE 1=1
AND wp_term_relationships.term_taxonomy_id IN (32)
AND wp_posts.post_type = 'post'
AND wp_posts.post_status = 'publish'
GROUP BY wp_posts.ID
ORDER BY wp_posts.post_date DESC
LIMIT 0, 12;

Unfortunately, that query’s execution plan cites Using temporary and Using filesort operations, which are both very expensive (the query engine must create temp table(s), sort the result, etc.).  The fact that the query was taking 1.5 seconds to return 12 records (on a cold query cache) supplements that claim.

Furthermore, something I questioned even before the execution plan was why the query made use of a group by clause on the table’s primary key w/out the use of any aggregate functions.  The join conditions and where clause used in the query assured a 1:1 join result, so using distinct, whether explicitly or implicitly via group by, wasn’t needed.  After removing it, I confirmed Using temporary and Using filesort were no longer in the execution plan:

As a result, we now have a query that performs 1,500 times faster.  The original query completed in about 1.5 seconds, whereas the revised query completes in less than a millisecond!  Even better news … after the optimization, the query is no longer reported in the MySQL slow query log.

Update: even though the optimization worked well here, it only worked part of the time in the production environment.  But on a positive note, an upgrade to MySQL 5.5 rendered the optimization obsolete.  Apparently the well known performance improvements in MySQL 5.5 were enough to fix this problem with no other effort required. Hooray for a MySQL 5.5 update!

Posted in Database, Performance | 1 Comment