A Final Blog Post

We will no longer be making posts to this blog, but we will be keeping it around so our readers can continue to access the archives.

We had originally intended to blog about the technical aspects of web design and development, while posting business related topics to the Intervals blog. But then we realized our audience was interested in both the technical and the business related topics. So we started getting more technical in our blog posts at Intervals.

From here on out we’ll continue blogging at the Intervals blog. Visit that blog for continued postings on everything related to web design, development, and creatives.

Thanks for reading all these years. We hope you may find some use still for our archived blog posts.

Photo credit: tiarescott

The Great 2011 SXSW Scavenger Hunt

This year the Pelago team will be attending the Interactive portion of South by Southwest in Austin, TX. This year will mark the fourth year that we have attended, and we’ve noticed that over the years the Interactive portion has lost much of its technological raison d’ĂȘtre and has become a bit more of a marketing wankfest wrapped up in a celebration of geek culture. Even SXSW itself seems to acknowledge this downward trend.

Granted, as I’ve often said, SXSW is an idea factory, and it really helps you stay abreast of the latest and greatest emerging trends in the tech industry. And if you dig hard enough, it is possible to find truly great panels with amazing speakers with tremendous insight to lend. But if you’re not careful, you might just end up in a panel that quickly devolves into an abyss of marketspeak douchebaggery (last year I was granted the special privilege of hearing from a panelist who was engaged in “hypervising neural networks”).

One of the emerging trends from this year’s panels appears to be games, so to help you make an exciting adventure of the highs and lows of this year’s SXSW Interactive, we’ve put together a list of items to keep an eye out for. Check off as many as you can, and post your results in the comment section of this blog post. The person with the most items checked off will win a free Intervals t-shirt. If you have any items to add to our list, let us know in the comments section.

So here, without further ado:

  • Someone very conspicuously using a new iPad2
  • Hipster wearing a vest or chunky Ray-Ban sunglasses
  • Someone wearing a fedora
  • Someone wearing a Threadless t-shirt
  • Someone wearing a ThinkGeek t-shirt
  • A dude with a HUGE (6+ inches) beard
  • Microsoft Silverlight promotional material clinging to its last vestiges of significance
  • Someone wearing more than one phone on their belt
  • Someone running Ubuntu on their laptop
  • A CR-48 Chrome notebook (the Google notebook)
  • A laptop with so many stickers, you can’t tell what brand it is
  • A 300 pound man using a tiny netbook
  • Free beer
  • Any mention of the following words from a speaker or panelist:
    • 4chan
    • Wikileaks
    • Libya/Egypt/Tunisia
    • Linux (or any Linux distro)
  • Someone taking notes with pen and paper
  • An attendee over 40
  • Pink fixed gear bicycle
  • Violet Blue sighting
  • Christopher Poole sighting (keynote doesn’t count)
  • A protester or soapbox evangelist carrying a sign

TIEBREAKER: A completely meaningless marketing buzzword/phrase (most syllables wins)

Let the games begin!

The Elusive Chrome 9 Blank Page Problem

AKA the images with negative margins problem.

Prior to our most recent update of Intervals we started to get bug reports from customers telling us that when they tried to view certain tasks, certain parts of the page would be blank. The reports ranged from the task summary being blank, up to nearly the entire page being blank. The only common thread to all these reports was that the users were using Chrome 9, which had just been released. It wasn’t appearing on any other pages, just the task view page. Here is how the page is supposed to look, as viewed in Firefox (you are looking at a development version of Intervals, so I apologize for all the weirdness):

And for contrast, here’s what the page looked like in Chrome:

As you can see, with the exception of the footer, the page was essentially blank. Using Chrome’s developer tools I dove into the DOM tree and tried to figure out what I could. For the most part, Chrome was calculating element positions correctly.

Something had to be throwing them off wildly, and that was indeed the case. A particular element was being calculated with a height of -214748328 pixels. As some of you may already know, that particular number is telling because it’s the minimum number that can be expressed in the signed 32 bit integer range. Chrome was, for all intents and purposes, calculating this element’s height as negative infinity.

A bit of further digging revealed the cause. The element in question contained an image with a height of 1 pixel and a margin (imposed by the stylesheet) of -1px -3px -2px -3px.

Essentially, the negative margin on the images threw off computation of the element’s dimensions to an unacceptable level and affecting all the surrounding elements. Removing the image solved the problem. Unfortunately, this particular block of HTML came from the customer. In fact, it came from an Outlook email client, was pulled in from their email account, and was created as a task. In this case, this particular image was used for email tracking (most of the time you don’t even notice these in the emails you read, but they’re quite common).

The solution in our case was to remove the negative margins being applied to these images from the stylesheet. It fixed the problem in this case. Incidentally, we ran into similar issues with images lacking a src attribute. We fixed that by adding the following line to our CSS file:

img:not([src]) { display: none; }

Secret Santa Name Picker Script

In the last ten years we’ve been gathering for our annual Pelago holiday party we’ve developed a fun Secret Santa tradition. The theme of the gift exchange is t-shirts. Each person picks a name from a hat to find out the lucky (or unlucky, depending on how you look at it) recipient of a hand-selected t-shirt. With one employee going remote and us being a technology company, it seemed somewhat Ludditean of us to continue picking strips of paper from a box.

This year we automated the process. We wrote a PHP script that would pick the names for us. It starts with an array of email addresses, matches them up in a random order and then emails each person their recipient. And it makes sure no one gets the same person. Take a look at the script below. It’s just a few simple lines of code. Feel free to use it for your next Secret Santa assignments. The basic code is below. Just plug the results in to your favorite email software to secretly distribute the results.


<?php

$emails = array(
     'name1@yourcompany.com',
     'name2@yourcompany.com',
     'name3@yourcompany.com',
     'name4@yourcompany.com',
     'name5@yourcompany.com',
);
$matches = array();

// randomize a key => value array
$givers = range(0,count($emails)-1);
$givees = range(0,count($emails)-1);

do {

     shuffle($givers);
     shuffle($givees);

     $matches = array_combine($givers, $givees);

     foreach ($matches as $er => $ee) {
          if ($er == $ee) continue 2;
     }

     break;

} while (true);

foreach ($matches as $er => $ee) {
     echo $emails[$er] . " gives to " . $emails[$ee] . "\n";
}

?>

Breaking the “rules” of web development

I’ve been developing for the web long enough — since my foray into PERL and HTML in 1994 — to know that there will always be a certain “rules” developers will adhere to when banging out lines of code. They are usually good rules because it is important for developers to follow them. However, developers need to know when to break from convention, when a higher purpose requires them to sacrifice the brilliance and elegance of their code. I personally have a few favorite examples from my own years of experience and from working with other developers…

The Almighty Framework

Frameworks are great for web development. The Model-View-Controller (MVC) framework, in its many interpretations for languages such as PHP, Ruby and Python, has made building web sites so much faster. And MVC frameworks make for nice clean code and a logical separation of data, business logic, actions and content. The view layer in and of itself is a godsend and the MVC framework has finally provided recognition for all the hard work put in by templating engine developers (i.e. Smarty).

What happens when the framework becomes a performance barrier? While developing our online project management software, Intervals, we’ve come across two main areas where we’ve had to lift the hood and require some of the framework internals.

First, web-based applications can be complex enough that the SQL queries can become cumbersome if they are not fine tuned. Using the default SELECT and JOIN conventions provided by the framework is not always ideal. When you start getting into the granular levels of optimizing SQL queries you have to get your hands dirty at the Model level. This means writing new queries, and tweaking and tuning them until they run as fast as possible. In some circumstances, this sometimes means associating a Model, especially list models, with a database table other than what it was intended. In addition, the framework is not going to optimize your database structure for you. Once the database is built you will need to tune the indices and learn about vacuuming and clustering.

Second, frameworks consume memory as they sift data up from the database, through layers of business logic, actions, and finally, into the view layer. For basic web-based applications that serve up limited information on a page, this is not a big concern and can usually be overcome using memcache if it does become one. However, if your web-based application is churning through a lot of data and presenting it to the user in real time, you will hit memory limits. This can happen, for example, with reports that contain a lot of data over a large date range (probably why Basecamp limits report data to a given number of months). In this case you will get the best performance with the View layer accessing the database directly using cursors. Yeah, I know developers won’t like this, but there comes a time when providing speed to your customers is more important than the framework upon which it is built. Anyways, cutting out the middle man you remove most of the strain on memory and increase the speed of the reports.

The Normalized Database

When we design databases our primary goal is to reduce the redundancy of data through the use of multiple tables, foreign keys, and queries that rely on JOINs. Developers will nitpick over a normalized database until every last bit of redundancy is ironed out. This approach works great for most web-based applications but when traffic increases all of those carefully crafted tables and keys, along with the JOIN-heavy queries begin costing you milliseconds, then seconds, before the app becomes unusable.

The solution is to begin denormalizing data. We called this a “necessary redundancy” at Pelago. You begin by identifying the slowest queries and removing their JOINs by placed the JOINed data in multiple tables. Than it’s up to stored procedures at the database level or developers at the code base level to make sure the redundant data is always kept redundant. The entire database doesn’t have to be denormalized all at once. Just the tables requiring JOINs that are causing you performance issues.

One Database to Rule Them All

Another harsh reality of web development is that the database may become too large and unwieldy for handling the number of people using the web-based application. Sys admins will start throwing around the four-letter word “sharding” as developers begin to cringe. If your app starts growing large enough, sharding may become a necessity for the app to scale. Breaking up your database onto multiple servers and keeping each copy of the database in sync with the others is a laborious task and should be a last resort. However, to dismiss sharding altogether in favor of throwing hardware at the problem is shortsighted. If your web-based application is growing you should be thinking about how you would shard the database if it becomes necessary in the long term. It’s better to have a plan in place before it’s needed than to be scrambling at the last moment to relieve an overloaded web application.

In fact, all of the “rules” I mentioned above should be addressed by web developers at some point if they have plans on scaling their web-based applications. Meanwhile, let’s hear from other web developers out there. What are some of the “rules” you’ve had to break?

Crunchbang Linux a Speedy Alternative to Ubuntu

Built using only Ubuntu, Crunchbang Linux has boiled down its distribution offering only the bare minimum needed to have a good balance between speed and functionality. Though not intended for older hardware, it is reported to work well in outdated environments where there is a premium placed on eking out as much speed as possible. Read more about it at http://crunchbanglinux.org/wiki/about and share with us any stories you might have in using it.

How to make Ubuntu Linux run faster on a laptop

If you google around on the web you will find there are several tutorials on how to make your Ubuntu Linux installation run faster — especially on older hardware. These tips are very useful and range from minor tweaks to major overhauls. Being an intermediate Linux user myself, I found some of the more difficult optimizations to be overwhelming and not something I would want to try at home — changes to how Linux writes to the hard drive, for example. While researching and implementing the optimizations I felt comfortable with on my older Dell laptop and my new Asus eeePC 901, I jotted down all of my notes for future reference. Below are some tips for the intermediate user on how to optimize your laptop (or desktop) running Ubuntu Linux.

 

Reduce Swappiness

Most laptops have enough RAM installed that the swap space on the hard disk shouldn’t really be used. Yet the default setting for swappiness in Ubuntu is 60. By lowering it to 10 we can reduce the number of read/writes the the hard disk. This is especially handy for netbooks with solid state drives in them. To reduce the swappiness, follow these steps:

  1. sudo sysctl -w vm.swappiness=10
  2. Add the following line to /etc/sysctl.conf
    vm.swappiness=10

 

Preload

Preload is an adaptive read-ahead daemon that monitors running applications and analyzes them for commonalities. It then uses this data to predict what applications you might run and preloads them into memory. The result is faster startup times for commonly used programs. Installing preload is as easy. Just run the following line in a terminal window:

sudo apt-get install preload

» For further explanation, read this article on loading applications quicker in Ubuntu using preload.

 

Boot-Up Manager

The Boot-Up Manager (BUM) is a useful utility for starting and stopping boot-up scripts. Once installed it will appear in the System -> Administration menu. You can use BUM disable unnecessary boot-up scripts that may be slowing down your boot time. For example, I disabled saned, an API library for scanners, because I know I’ll never be attaching a flatbed scanner to my laptop. To install BUM:

sudo apt-get install bum

» Read the documentation for Boot-Up Manager

 

Startup Applications

If you are using Gnome for your desktop manager there will be an option to select which applications are started when you login. Go to System -> Preferences -> Startup Applications and uncheck the programs you don’t need. For example, I disabled the Evolution Alarm Notifier because I don’t use Evolution for setting alarms. I also disabled the Remote Desktop server, UME Desktop Launcher, and UNR Launcher. Just be careful to uncheck them and not remove them. In the case that you’ve removed something critical, you’ll want to be able to get it back.

 

Replace Metacity with Openbox

Openbox is a minimalistic and lightweight window manager that is known to run much faster than metacity, its bulkier counterpart. And you can easily get it working while still using Gnome as your window manager / desktop environment. By running Openbox inside the Gnome environment your desktop will become cleaner and faster. To install it, follow these steps:

  1. Install openbox using:
    sudo apt-get install openbox obconf openbox-themes
  2. Setup openbox as the default window manager by adding an entry in Startup Applications.
    To do this, go to System -> Preferences -> Startup Applications and enter the following:

    • Name: Openbox
    • Command: openbox ––replace

    Note: According to the Openbox documentation you can make it the default by choosing the GNOME/Openbox session when logging in and saving this as your default session. However, this functionality is broken in 9.04. The above steps are a workaround until this is fixed.

  3. Choosing the Openbox theme and other configuration settings
    Go to System -> Preferences -> Openbox Configuration Manager to choose a theme you like and to update other settings such as Appearance and Windows.
    Note: changing the Desktops setting doesn’t effect the Gnome applet controlling the number of desktops. To change the number of desktops, you will need to revert back to Metacity and change them, then re-enable Openbox

» More information, documentation and screenshots available at the Openbox web site

 

That’s all for now

I found that making these few simple changes decreased load times on my older laptop significantly and made my newer netbook more minimal. Hopefully, these tips will help you as well.

 

Face detection in pure PHP

Face detection in pure PHP

Technical description.

PHP crons, linux, and the hostname

When running PHP as a cron, the $_SERVER['HOSTNAME'] variable is not set, nor are any other variables that will identify which server your cron is running on. This can be problematic if you are running the same cron on multiple servers, such as in a load balanced environment, and you need the cron to report back or log information about the server on which it ran.

Here is some code for getting the hostname value from your linux network configuration, assuming you have setup /etc/sysconfig/network properly.


//get hostname info from /etc/sysconfig/network
preg_match('/HOSTNAME=(.*)/', file_get_contents('/etc/sysconfig/network'), $network);
$hostname = split("\=", $network[0]);
echo $hostname[1]; //this equals the value of your HOSTNAME

ISO 8601 Date Validation That Doesn’t Suck

UPDATED February 19th, 2010: As BobM pointed out, the original solution to this problem didn’t account for fractional decimals. Originally I didn’t include them because Intervals didn’t require that level of precision, but apparently fractional decimals are quite common elsewhere. Because of that, I’ve updated this post, along with the regex, to include support for fractional decimals.

For the Intervals API, we’re wrestling with issues surrounding data input validation. This recently became interesting when the matter of date validation came up. Ordinarily, Intervals allows many, many different date formats, dependent on the locale that the customer is using (for example, Intervals may expect the date format ‘mm/dd/yyyy’ for US customers, ‘dd.mm.yy’ for a customer in Austria).

For our API developers, we wanted to use a common, universal format, one that would be easily compatible with our application and database layers. For that we selected ISO 8601, which is great in terms of widespread use, but not so great in terms of how complicated its specifications are.

Generally, ISO 8601 looks something like ’2009-05-20′ for dates and ’2009-05-20 12:30:30′ for date/time combinations. These two examples encompass 98% of the user input we’re likely to encounter. But we wanted to make sure that if we told developers they could use ISO 8601 dates, our system would support it. Read the rest of this entry »