Minify and consolidate JavaScript scripts via PHP wrapper

Posted: March 16, 2010 in HTTP, Javascript, PHP, Web language, Zend
Tags: , , , , ,

The goal of this article is to provide details about the implementation I use on my current project to compress (actually minify) and consolidate (aggregate) JS files into one <script> call. Another issue I had was to deal with the web browser’s local cache and the fact that we need to update our javascript code. It is not really user friendly to ask our clients to remove their cache neither press ‘Ctrl + F5’ Keys!

My ultimate goal was to get a “A” grade a the Yahoo! YSlow Firefox plug-in. Here are their rules: Best Practices for Speeding Up Your Web Site. All I can say is… I’ve reach this goal!

I saw the post “Compresser et ranger son CSS avec PHP“, and decided to post the solution I use for the 2 projects I work on.

You can get all the sources described and explained in this article here: minify_js.zip.

Gain expected

By using this JS wrapper and the appropriate apache setup, users have a better experience of the web site because pages are loaded quicker and the JS code is also quickly interpreted (almost no spaces nor comments remain in the javascript minified files).

What you should know

To make this code works, some HTTP skills is required (404 error pages), PHP code is simple (only static methods are used). The only issue may be with the web server setup. As you’ll see below, you may need to modify the apache httpd.conf file (or IIS setup).

How it works?

  1. The first step is to create minified JS sources. It will produce ‘.min.js’ files based upon the original ‘.js’ files. This is done via a custom makefile.
  2. In the PHP views, instead of writing <script> HTML tags, I call a kind of view’s helper (not really because it is a static call: works every where, and do not require the MVC model of Zend Framework to work. This “helper” queues JS scripts.
  3. When I want to print out the <script> HTML tags, I call the printer of the above class. Il will produce the minimum HTTP calls.
  4. 404 error are caught thanks to the ‘.htaccess’ file and rerouted to a PHP script that creates the aggregates files and sends it. On next calls it will be directly delivered by the web server, without any PHP computes.

Minify JS files

For each .js files, create a minified version. As said above, I use yuicompressor:

yuicompressor --type js --charset iso-8859-1 jquery.js > jquery.min.js

I use this alias: (change the path according to your own setup!)

alias yuicompressor='java -jar /usr/local/lib/yuicompressor-2.4.2/build/yuicompressor-2.4.2.jar'

Index file

Changes in your code

Instead of writing<script> HTML tags, you will use the “Ihm_Js::addScript()” wrapper (see lines 24~32 above). Once your list is complete, simply write out where you want (header or footer) with: “echo Ihm_Js::get();” (see line 40 above).

Source code

<!--?php <br ?-->/**
 * JS_Minify tutorial
 *
 * @package   Core
 * @version   $Id: Env.php 4282 2010-03-12 14:06:55Z wloche $
 * @author    Wilfried Loche <wloche@hotmail.com>
 * @copyright Copyright (c) 2010, Wilfried Loche
 * @license   https://wloche.wordpress.com/2010/03/15/new-bsd-license/ New BSD License
 */

set_include_path(
    realpath(dirname(__FILE__) . '/../library/')
    . PATH_SEPARATOR . get_include_path()
);

//ini_set('display_errors', 'On');

/** Common environment for the project */
require_once 'Env.php';
/** JS loader */
require_once 'Ihm/Js.php';

//--- Add common JS
Ihm_Js::addScript('jquery.js');
Ihm_Js::addScript('jquery-ui.custom.js');
Ihm_Js::addScript('ui.datepicker.js');

Ihm_Js::add(
    '$(document).ready(function(){ $("#datepicker").datepicker(); });',
    Ihm_Js::END
);
?>


<meta http-equiv="content-type" content="text/html; charset=iso-8859-1" />
Minify and consolidate JavaScript scripts
    	<link href="http://jqueryui.com/latest/themes/base/jquery.ui.all.css" rel="stylesheet" type="text/css" />
    <!--?php echo Ihm_Js::get(); ?--></pre>
<h3>Minify and consolidate JavaScript scripts</h3>
<pre>

Have a look a the source's page: you will find only one JS <script> HTML tag that produce only on HTTP call.

To check JS scripts still work fine, you should see a beautiful calendar!</pre>
<div id="datepicker"></div>
<pre>

  

Screenshot

jQuery Calendar

JS minify and aggregate screens

Yahoo… it still work! The 3 javascript’s files put into just one minified file is correctly interpreted by the browser!

Generated HTML code

Simply check that you JS code is still correctly interpreted!

<script type="text/javascript" src="scripts/1.0.0.__REVISION__/jquery,jquery-ui.custom,ui.datepicker.js" language="Javascript"></script><script type="text/javascript" language="JavaScript">// <![CDATA[
$(document).ready(function(){ $("#datepicker").datepicker(); });
// ]]></script>

As you can see, there are two <script> HTML tags, one for the scripts, and one another for the javascript code.

.htaccess

Here is the public/scripts/.htaccess content file:

RewriteEngine off
ErrorDocument 404 /URL_to_scripts_folder/scripts.php

To get it work on apache (I don’t know how to setup a IIS web server to catch 404 errors), this option has to bet set:

AllowOverride all

Note: allow .htaccess makes a lot of I/O file accesses. It is recommanded to put the per folder setup directly in the main httpd.conf file (the last point I have to work on with production environment!).

JS aggregator

Goal

This script is in charge to create the aggregate javascript file with the security this kind of script requires.
It will receive the scripts via the $_SERVER[“REDIRECT_URL”] variable.

It does:

  1. explode the $_SERVER[“REDIRECT_URL”] with ‘,’ to get all JS files we want in our web page,
  2. depending on the Ihm_Js::MINIFY, use the .min.js or the .js files,
  3. if not already done, create the folder named Env::getVersion() to store aggregated JS files,
  4. get the JS files contents,
  5. write the file so the next call will not require any PHP load,
  6. output the content.

Source code

Here is the public/scripts/scripts.php content file:

<!--?php <br ?-->/**
 * Generate JS files dynamically
 *
 * This script is called by a 404 error page catched by the web server.
 *
 * Here is the .htaccess for apache:
 * <code>
 * RewriteEngine off
 * ErrorDocument 404 /URL_to_scripts_folder/scripts.php
 * </code>
 *
 * To get the .htaccess file read by apache, you may need to activate this option:
 * <code>
 * AllowOverride all
 * </code>
 *
 * @package    IHM
 * @subpackage JS
 * @version    $Id: scripts.php 3838 2009-12-22 14:34:31Z wloche $
 * @author     Wilfried Loche <wloche@hotmail.com>
 * @copyright  Copyright (c) 2010, Wilfried Loche
 * @license    https://wloche.wordpress.com/2010/03/15/new-bsd-license/ New BSD License
 */

if (!defined('__DIR__')) {
    /** __DIR__ Constant, waiting for PHP 5.3! */
    define('__DIR__', dirname(__FILE__));
}

set_include_path(
    realpath(__DIR__ . '/../../library/')
    . PATH_SEPARATOR . get_include_path()
);

/** Common environment for the project */
require_once 'Env.php';
/** JS loader */
require_once 'Ihm/Js.php';

$infos = pathinfo($_SERVER["REDIRECT_URL"]);

$basename = rtrim($infos['basename'], '.js');
$jsFiles = explode(',', $basename);

$jsToInclude = array();
foreach ($jsFiles as $jsFile) {</pre>
<div id="_mcePaste">//--- Sanitize requested JS files</div>
<pre>
$jsFile = preg_replace('/[^a-zA-Z0-9_\.\-]/', '', $jsFile);

    $originalScriptPath = __DIR__ . '/' . $jsFile . '.js';
    if (Ihm_Js::MINIFY) {
        $minScriptPath      = __DIR__ . '/' . $jsFile . '.min.js';
        if (!file_exists($minScriptPath)) {
            if (!file_exists($originalScriptPath)) {
                trigger_error("JS file '{$originalScriptPath}' does not exist", E_USER_WARNING);
                header("HTTP/1.0 404 Not Found");
                exit();
            } else {
                $jsToInclude[] = $originalScriptPath;
            }
        } else {
            $jsToInclude[] = $minScriptPath;
        }
    } else {
        if (!file_exists($originalScriptPath)) {
            trigger_error("JS file '{$originalScriptPath}' does not exist", E_USER_WARNING);
            header("HTTP/1.0 404 Not Found");
            exit();
        } else {
            $jsToInclude[] = $originalScriptPath;
        }
    }
}

if (!file_exists($originalScriptPath) || $infos['extension'] != 'js') {
    header("HTTP/1.0 404 Not Found");
    exit();
}

//--- Check the version waited for
$askedVersion = basename($infos['dirname']);

$currentVersion = Env::getVersion(Env::TEXT);
if ($currentVersion !== $askedVersion) {
    trigger_error("Version '$askedVersion' asked, current is '$currentVersion'", E_USER_WARNING);
    header("HTTP/1.0 404 Not Found");
    exit();
}

//--- Create version 's dir if necessary
if (!file_exists($currentVersion)) {
    mkdir($currentVersion);
}

ob_start();

print("/** Generated on " . date('c') . " */\n\n");

/** contenu de la page */
foreach ($jsToInclude as $jsFile) {
    echo "\n/* ### " . basename($jsFile) . " ### */\n\n";
    /** contenu de la page */
    include($jsFile);
}

$out = ob_get_clean();

header("HTTP/1.0 200 Found");
header('Content-Length: ' . strlen($out));
header('Content-type: text/javascript; charset=iso-8859-1');
$expireAt = gmdate(DATE_RFC822, time() + 3600);
header("Expires: $expireAt");
header("Last-Modified: " . gmdate("D, d M Y H:i:s") . " GMT");
header("Pragma: public");

print($out);

//--- Ecrit le résultat pour la prochaine lecture du fichier JS directement
file_put_contents(__DIR__ . '/' . $currentVersion . '/' . $infos['basename'], $out, LOCK_EX);
$jsLocation = __DIR__ . '/' . $currentVersion . '/' . $infos['basename'];

Generated files

Files are generated here : public/scripts/1.0.0.__REVISION__/. This last folder is given by the function Env::getVersion(Env::TEXT) described in the next tools section.

This tutorial has writen this file: jquery,jquery-ui.custom,ui.datepicker.js. Here is the beginning of the file:

/** Generated on 2010-03-15T22:34:33+01:00 */

/* ### jquery.min.js ### */

/*
 * jQuery JavaScript Library v1.3.2
...

Tools classes

Env class

Goal

Provide some environment’s informations, such as the version of the product that will be updated at each product release. Another information could be to know weather the current environment is production or not. If not, you may not want to write JS files.

Source code
<!--?php <br ?-->/**
 * Common environment for the project
 *
 * @package   Core
 * @version   $Id: Env.php 4282 2010-03-12 14:06:55Z wloche $
 * @author    Wilfried Loche <wloche@hotmail.com>
 * @copyright Copyright (c) 2010, Wilfried Loche
 * @license   https://wloche.wordpress.com/2010/03/15/new-bsd-license/ New BSD License
 */

/**
 * Common environment for the project
 *
 * @package   Core
 * @author    Wilfried Loche <wloche@hotmail.com>
 * @copyright Copyright (c) 2010, Wilfried Loche
 * @license   https://wloche.wordpress.com/2010/03/15/new-bsd-license/ New BSD License
 */
class Env
{

    /** Actual version */
    const VERSION = '1.0.0';
    /** Actual revision (may be automatically updated by a release script) */
    const REVISION = '__REVISION__';

    /** For HTML display */
    const HTML = 'H';
    /** For text display */
    const TEXT = 'T';

    /**
     * Get the actual version
     *
     * @param char $mode Mode among {@link HTML} and {@link TEXT}
     *
     * @return string
     */
    public static function getVersion($mode = self::HTML)
    {

        if ($mode != self::HTML && $mode != self::TEXT) {
            throw new Exception('mode must be either HTML or TEXT');
        }

        $version = self::VERSION;
        switch ($mode) {
            case self::HTML:
                $version .= '<!-- (' . self::REVISION . ') -->';
                break;

            default:
                $version .= '.' . self::REVISION;
                break;
        }

        return $version;
    }

}

JS wrapper class

Goals

This class is a simple wrapper for JS calls. It produces the HTML code to:

  1. minimize the HTTP calls,
  2. and use the current project’s version, so that you can take full advantages of the client’s web browsers’ cache.
Source code
<!--?php <br ?-->/**
 * JS loader
 *
 * @package    IHM
 * @subpackage JS
 * @version    $Id: Js.php 3822 2009-12-21 09:38:24Z wloche $
 * @author     Wilfried Loche <wloche@hotmail.com>
 * @copyright  Copyright (c) 2010, Wilfried Loche
 * @license    https://wloche.wordpress.com/2010/03/15/new-bsd-license/ New BSD License
 * @link       http://framework.zend.com/manual/fr/zend.view.helpers.html#zend.view.helpers.initial.headscript
 */

/** Common environment for the project */
require_once 'Env.php';

/**
 * JS loader
 *
 * @package    IHM
 * @subpackage JS
 * @author     Wilfried Loche <wloche@hotmail.com>
 * @copyright  Copyright (c) 2010, Wilfried Loche
 * @license    https://wloche.wordpress.com/2010/03/15/new-bsd-license/ New BSD License
 */
class Ihm_Js
{

    /** Aggergate JS calls, shoud be put in a config file */
    const AGGREGATE = true;

    /** Use minified files? */
    const MINIFY = true;

    /** @var array JS code */
    private static $_js = array(
        self::BEGIN  => array(),
        self::MIDDLE => array(),
        self::END    => array()
    );

    /** @var array JS scripts */
    private static $_jsExt = array(
        self::BEGIN  => array(),
        self::MIDDLE => array(),
        self::END    => array()
    );

    /** @var array JS code that will be executed when the page is load */
    private static $_jsOnLoad = array(
        self::BEGIN  => array(),
        self::MIDDLE => array(),
        self::END    => array()
    );

    /** Scripts will be added at the beginning */
    const BEGIN  = 0;
    /** Scripts will be added in the middle */
    const MIDDLE = 50;
    /** Scripts will be added at the end */
    const END    = 100;

    /** Codes JS internes et Scripts externes */
    const TYPE_ALL = 'ALL';
    /** Scripts externes */
    const TYPE_SCRIPT = 'SCRIPT';
    /** Scripts internes */
    const TYPE_CODE = 'CODE';
    /** Scripts internes sur chargement de la page */
    const TYPE_CODE_ONLOAD = 'CODE_ONLOAD';

    /**
     * Pure static class
     */
    final private function __construct()
    {
    }

    /**
     * Pure static class
     */
    final private function __clone()
    {
    }

    /**
     * Add a JS code
     *
     * @uses $_js
     *
     * @param string $jsCode   JS Code
     * @param string $key      Key
     * @param string $position Position
     */
    static function add($jsCode, $key = null, $position = self::MIDDLE)
    {
        if ($key !== null) {
            self::$_js[$position][$key] = $jsCode;
        } else {
            self::$_js[$position][] = $jsCode;
        }
    }

    /**
     * Add a JS code that will be executed once the page is fully loaded
     *
     * @uses add()
     *
     * @param string $jsCode   Code JS
     * @param string $key      Clé
     * @param string $position Position
     */
    static function addOnLoad($jsCode, $key = null, $position = self::MIDDLE)
    {
        if ($key !== null) {
            self::$_jsOnLoad[$position][$key] = $jsCode;
        } else {
            self::$_jsOnLoad[$position][] = $jsCode;
        }
    }

    /**
     * Add a JS script
     *
     * Duped calls are not taken.
     *
     * @uses $_jsExt
     *
     * @param string $jsScript JS script
     * @param string $position Position
     */
    static function addScript($jsScript, $position = self::MIDDLE)
    {
        self::$_jsExt[$position][$jsScript] = true;
    }

    /**
     * Remove all added scripts
     *
     * @uses $_jsExt
     *
     * @param string $position Position
     *
     * @throws Exception
     */
    static function removeScripts($position = '')
    {
        if ($position != '') {
            if (!isset(self::$_jsExt[$position])) {
                throw new Exception("$position is not allowed");
            }
            $pos = array($position);
        } else {
            $pos = self::$_jsExt;
        }

        foreach ($pos as $position) {
            self::$_jsExt = array();
        }
    }

    /**
     * Get actual JS codes
     *
     * @uses $_js
     * @uses $_jsExt
     *
     * @param string $type What to get, among:
     *                     - {@link TYPE_ALL},
     *                     - {@link TYPE_SCRIPT},
     *                     - {@link TYPE_CODE},
     *                     - {@link CODE_ONLOAD}.
     *
     * @return string
     */
    static function get($type = self::TYPE_ALL)
    {

        if (!count(self::$_js) && !count(self::$_jsExt)) {
            return '';
        }

        $html = '';
        $jsUri = 'scripts/' . Env::getVersion(Env::TEXT) . '/';
        foreach (array(self::BEGIN, self::MIDDLE, self::END) as $position) {

            //--- Scripts
            if (($type == self::TYPE_ALL || $type == self::TYPE_SCRIPT)
                && isset(self::$_jsExt[$position]) && count(self::$_jsExt[$position])) {

                if (self::AGGREGATE) {
                    foreach (self::$_jsExt[$position] as $jsScript => $value) {
                        $jsList[] = $jsScript;
                    }
                    $generatedFilename = implode(',', $jsList);
                    $generatedFilename = str_replace('.js', '', $generatedFilename);
                    $html .= '<script type="text/javascript" src="' . $jsUri . $generatedFilename . '.js" language="Javascript"></script>' . "\n"; } else { foreach (array_keys(self::$_jsExt[$position]) as $jsScript) { $html .= '<script type="text/javascript" src="' . $jsUri . $jsScript . '" language="Javascript"></script>' . "\n";
 }
 }
 self::$_jsExt[$position] = array();
 }

 //--- Code JS
 if (($type == self::TYPE_ALL || $type == self::TYPE_CODE)
 && isset(self::$_js[$position]) && count(self::$_js[$position])) {

 $html .= '<script type="text/javascript" language="JavaScript">// <![CDATA[
';
                $html .= implode("\n", self::$_js[$position]);
                $html .= '
// ]]></script>' . "\n";

 self::$_js[$position] = array();
 }
 }

 //--- Code JS sur chargement de la page
 if (($type == self::TYPE_ALL || $type == self::TYPE_CODE_ONLOAD)
 && (count(self::$_jsOnLoad[self::BEGIN])
 || count(self::$_jsOnLoad[self::MIDDLE])
 || count(self::$_jsOnLoad[self::END]))) {

 $html .= '<script type="text/javascript" language="JavaScript">// <![CDATA[
';
            $html .= "Event.observe(window, 'load', function() {\n";

            foreach (array(self::BEGIN, self::MIDDLE, self::END) as $position) {
                if (!isset(self::$_jsOnLoad[$position]) && count(self::$_jsOnLoad[$position])) {
                    continue;
                }

                $html .= implode("\n", self::$_jsOnLoad[$position]);

                self::$_jsOnLoad[$position] = array();
            }
            $html .= "});\n";
            $html .= '
// ]]></script>' . "\n";
 }

 return $html;
 }

}

Final apache setup

To take full advantages of the web browsers’ cache, here is an extract of the apache setup I use in production.

Compression

Use gzip compression the most you can. All web browsers now fully support it. You will save a huge internet bandwidth based on a minor CPU coast.

You will notice some restrictions for images and old browsers.

# http://httpd.apache.org/docs/2.0/mod/mod_deflate.html
# Compress everything except images

  # Insert filter
  SetOutputFilter DEFLATE

  # Netscape 4.x has some problems...
  BrowserMatch ^Mozilla/4 gzip-only-text/html

  # Netscape 4.06-4.08 have some more problems
  BrowserMatch ^Mozilla/4\.0[678] no-gzip

  # MSIE masquerades as Netscape, but it is fine
  # BrowserMatch \bMSIE !no-gzip !gzip-only-text/html

  # NOTE: Due to a bug in mod_setenvif up to Apache 2.0.48
  # the above regex won't work. You can use the following
  # workaround to get the desired effect:
  BrowserMatch \bMSI[E] !no-gzip !gzip-only-text/html

  # Don't compress images
  SetEnvIfNoCase Request_URI \
    \.(?:gif|jpe?g|png|pdf)$ no-gzip dont-vary

  # Make sure proxies don't deliver the wrong content
  #Header append Vary User-Agent env=!dont-vary


ETags

Get a unique ID for files, even with load balanced servers (do not use inode information!).

# http://httpd.apache.org/docs/2.2/mod/core.html
FileETag MTime Size

Note: to get it working in load balanced environment, you need to deploy files via a zip or tar. So that modification dates will be the same on all your web servers.

Expires

The expire information can be set by default for all static files (images, css and js files). The internet explorer will not try to get the file until it is not expired.

# http://httpd.apache.org/docs/2.2/mod/mod_expires.html
# ExpiresByType image/gif "access plus 14 days"

ExpiresActive On
ExpiresDefault "access plus 1 week"

Conclusion

Cons

As usual, it is difficult to get a better user experience with a better developer experience! To debug/code a javascript file, you’ll need to first use original files. When your code is tested, you’ll have to minify it and test with the end-user setup (use minified and aggregate files to be sure it will work also for end-users), finally, commit the original and minified files the same time.

For sure, the testing phase is a bit longer than earlier but the developers of my team agreed with 2 facts:

  1. it’s not so boring!
  2. the end-user experience is the (almost ;)) only goal with have!

Finally

I hope this article will help you to get a faster web experience for your users. Feel free to ask me any questions/comments. This article is the result of many years of web projects I’ve worked on and still work on.

About me

Wilfried Loche, a web developper/architect of two projects and project leader of Concerto for ADP France, and more specially for small and medium businesses. Sorry for my English, you surely have noticed I’m French. I haven’t speak English at work for many years now…

Since Jul 21st, 2011, I am a Zend Certified Engineer.

Advertisements
Comments
  1. doms says:

    That looks very nice. Before that I’ve mainly focused on optimisations of php code in order to lighten charge on server. I now realize that an approach like that will also reduce number of requests on my server and so could partially solve overload problems.

  2. wloche says:

    Remove internal ADP paths for yuicompressor (thx to Séb)

  3. wloche says:

    Publish the ZIP containing all sources: http://tinyurl.com/yytzb5u

  4. Really nice blog for learning php script and give information how to use php script and also java script.

  5. christianity says:

    I am actually thankful to the holder of this website who has shared this enormous
    paragraph at here.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s