URL of the article:

Issue: Special.2004
Benchmarking PHP with no BS
Understanding the Performance Characteristics of PHP
by John Lim
This article focuses on a rather difficult topic - understanding the performance characteristics of PHP. It is difficult, not because PHP is complex, but because PHP's performance depends on many outside factors such as software versions, available hardware, operating system settings, and network configuration.

Writing high performance PHP code involves many factors. Some of the more important ones include:
    (a) Choosing appropriate algorithms (b) Understanding the performance characteristics of PHP (c) Tuning your application subsystems, such as the operating system, the caching sub-system, the network, and your database
To make sense of all these factors, I have created the PHP Benchmarking Suite (PHP BS in short) that measures some common PHP coding techniques. The numbers generated by the benchmarking suite may not be important in themselves, but they could serve as a useful guide to the relative merits of particular techniques. You are also advised not to put too much faith in this article's recommendations, as it is possible that improvements in PHP could render the advice obsolete. It is probably better to run the benchmarking suite and decide for yourself, based on the specifics of your system configuration.

General Advice
PHP is a scripting language that has been designed to favour ease-of-use over performance. This however, does not mean that PHP is slow. Much of the important functionality of PHP is written as high-speed C extensions. As you will see in the benchmarks, the fastest PHP code tries to make use of extension functionality as much as possible. Further, due to PHP's dynamic nature, PHP scripts are recompiled every time they are invoked. The most important optimisation you can perform is to avoid this recompilation by installing an opcode cache such as Zend Accelerator or Turck MMCache. You will typically get a 250% or more increase in performance for short-lived PHP scripts. An understanding of how PHP internally stores variables is also helpful in analysing the benchmarks. PHP variables are stored in a C data structure called a zval (described in zend.h). For integers and floats, the numeric value is stored in the zval itself. For arrays, strings and objects, additional storage has to be allocated outside of the zval. This means that benchmarks involving numeric variables should run faster than ones with string, object, or array variables.

An intro to Opcode Cache from Got Boost - Opcode Cache Shootout
by Ilia Alshanetsky (Issue 06.2003, International PHP Magazine)
One way to make PHP scripts run faster is to meticulously profile and optimise your code utilizing various tricks and techniques. Certainly a good idea once in a while, however, in most cases this approach is way too time consuming and labour intensive, making it a non-option in an environment where deadlines are short and customers need their applications working yesterday. Fortunately, PHP's design leaves a nice 'loophole' for a rather neat optimisation that can be applied almost instantly for a meagre cost of a few megabytes of random access memory (RAM). This 'loophole' is filled by an opcode cache that can often magnify the performance of a script several times. Instead of parsing a script every single time it is executed, opcode cache does it only once and stores the generated opcodes inside shared memory for future use. The next time a script is executed it checks if the script has changed. If it hasn't, instead of parsing, opcode cache simply fetches the pre-generated opcodes from shared memory (SHM) and passes them on to the Zend Engine to execute. This way, not only do you benefit from elimination of the parsing step, but you also gain the performance boost of not having to read the script and its associated files from the hard drive - the data is read from memory which is much faster than disk access.

Installing and Running the BS System
To install the suite, unpack the archive into a directory accessible by your Web server. Then open index.php from a Web browser. You will see a list of tests to run. Select one of the tests, and the benchmark results will be displayed. The benchmark results follow a standard format, as shown in the following table. Each test file contains two benchmark functions: bench1() and bench2(). Both functions are run 1000 times, and the total execution time for each function is shown in seconds. In all benchmarks, faster is better.

Back    Run bench2() first.
Source    Test Once    Reading CONSTANTS Versus Global Variables
Constants x 2.40 faster 0.0005 Globals 0.0012

You can view the source code (click on Source), or verify the benchmark results (Test Once). Test Once displays any output returned by the benchmark functions, so you can verify that the functions were written correctly. Normally bench1() is run first. You can switch the order of execution [run bench2() first] if you suspect that this skews the benchmark results.

Benchmark Platforms
All tests were done on 3 machines.
  • PHP 4.3.3 and PHP 5b4 running on Red Hat 8.0 Linux SMP Server with 2 x 1Ghz Pentium 3 and 1 Gb RAM.
  • PHP4.3.3 and PHP5b4 running on Windows XP Professional with 2.5 Ghz Intel Pentium 4 and 0.5 Gb of RAM.
  • PHP 4.0.6 running on SuSE Linux 7.0 with P133 MHz, 128 Mb of RAM.
Zend Optimizer 2.1.0 and Turck MMCache 2.4.46 were running when PHP 4.3.3 was installed, unless otherwise mentioned.

Exceptions Suite
These tests can only be run on PHP5b4, and are meant to test exception overhead. By default, the total time in seconds for 1000 executions is displayed.

Source    Test Once    Testing PHP 5 exceptions (with 1 throw)
No exceptions x26.71 faster 0.0007 With exceptions - throw 1 exception 0.0187


Source    Test Once    Testing PHP 5 exceptions (no throw)
No exception code x 1.71 faster 0.0007 With exception code which is never invoked 0.0012

In the first test, we measure the cost of throwing an exception nested 1 function deep as (0.0187-0.0007 seconds) / 1000 executions = 18 microseconds. In the second test, the overhead of the exception code, if no exception is thrown, is roughly (0.0012-0.0007 seconds) / 1000 executions = 0.5 microseconds. In general, given that the typical PHP script takes about 50,000 - 100,000 microseconds (0.05 to 0.10 seconds) to execute, we can safely assume that the overhead of exceptions is extremely low.

Loops Suite
Software programs spend most of their time in loops, so loops need to be highly optimised. In PHP, I believe that the most common looping operation is to perform some operation on each element of an array. Since PHP provides several techniques for doing this, it makes choosing the optimal method extremely confusing. In this test suite, a 50 element array $ARR, indexed from 0 to 49, is processed using different methods. In the first set of tests, we process $ARR when the array contains integers:

Method ordered by PHP 4.3.3 Performance PHP4.3.3 PHP5b4
for ($i=sizeof($ARR); --$i>=0;); 0.0745 0.0535
$v = reset($ARR); do { } while ($v = next($ARR)); 0.0752 0.0606
array_map() 0.0755 0.0644
foreach($ARR as $v); 0.0817 0.0604
for ($i=0, $max=sizeof($ARR); $i < $max; $i++); 0.0874 0.0665
foreach($ARR as $k => $v); 0.1052 0.0691
reset($ARR); while(list(,$v) = each($ARR)); 0.1262 0.1047
reset($ARR); while(list($k,$v) = each($ARR)); 0.1415 0.1179

In the second set of tests, we process $ARR when the array contains strings:

Method ordered byPHP 4.3.3 Performance PHP4.3.3 PHP5b4
for ($i=sizeof($ARR); --$i>=0;); 0.0735 0.0533
foreach($ARR as $v); 0.0791 0.0605
array_map() 0.0808 0.0594
for ($i=0, $max=sizeof($ARR); $i < $max; $i++); 0.0866 0.0643
$v = reset($ARR); do { } while ($v = next($ARR)); 0.0984 0.0706
foreach($ARR as $k => $v); 0.1016 0.0689
reset($ARR); while(list(,$v) = each($ARR)); 0.1233 0.1056
reset($ARR); while(list($k,$v) = each($ARR)); 0.1412 0.1176

A quick review of the above figures demonstrates that PHP 5, despite its beta status, is much faster in loop execution. The above measurements reveal that the fastest way of processing a numerically indexed array is:

for ($i=sizeof($array)
; --$i>=0 ; )
process_element($array[$i]);

Most programming languages are tuned for very fast evaluation of zero and non-zero values, and the looping condition here (--$i>=0) takes advantage of this fact. This technique is particularly useful when the processing order of the elements is not important. For example, when you are calculating the average of an array of numbers. Some programmers might find the above construct ugly. In fact, there is nothing unusual about this technique; it is a standard pattern for high performance loops among C programmers. The foreach loop has pretty good performance too. It came 2nd for string processing and is reasonably fast when handling numbers. In general, I would stick to using foreach because I feel code clarity is more important, with judicious use of the faster for loop technique (;--i >= 0;) only when speed is really essential.

Regular Expression Suite
The following figures below are taken from PHP4. PHP5b4 results are similar:

Source    Test Once    Searching for substring With strpos Versus perl regex
strpos x 2.50 faster 0.0010 preg_match 0.0025


Source    Test Once    Searching for substring With perl regex And ereg
preg_match x 1.79 faster 0.0024 ereg 0.0043

In these tests, we find that searching for a substring is fastest using strpos (2.50 times faster than perl-style regular expressions). And perl-style regular expressions are substantially faster than POSIX regular expressions (ereg).

Variable Suite
This is a mixed bag of tests. The results are taken from PHP4. We will highlight results that differ in PHP5b4. The first test measures whether accessing constants or global variables is faster. In PHP 4.2.3, globals were faster. In PHP4.3.3 and PHP5b4, the reverse is true.

Source    Test Once    Reading CONSTANTS Versus Global Variables
Constants x 2.40 faster 0.0005 Globals 0.0012

The next tests check whether $s = $s + 1 or $s += 1 is faster. With an optimizing compiler such as Zend Optimizer installed, there is no difference in performance between + and += because $s = $s + 1 is converted to the simpler $s += 1. This is demonstrated by the fact that the winner of repeated test-runs randomly oscillates between + and +=. But with the Zend Optimizer removed, the results consistently show that += is faster, as shown next:

Source    Test Once    + vs += Integer operators
+ integer 0.0015 += integer x 1.36 faster 0.0011


Source    Test Once    + vs += String operators
+ strings 0.0023 += strings x 1.21 faster 0.0019


Source    Test Once    += vs + String operators
+= strings x 1.20 faster 0.0020 + strings 0.0024

The last benchmark in this section measures the overhead in invoking a variable with a short name and one with a long name. In both PHP4 and PHP5b4, the one character variable $a is faster. This suggests that the internal variable hashing algorithms can be further tuned.

Source    Test Once    short Versus long variable names
short var name (1 char) x 2.67 faster 0.0003 long var name (72 chars) 0.0008

Functions Suite
This tests various parameter-passing techniques. The next two results test passing parameters by value and by reference when calling a function. In PHP4, for both arrays and objects, passing by reference is substantially faster.

Source    Test Once    Arrays: Call by ref Versus Call by value
Call by Ref x17.38 faster 0.0013 Call by Value 0.0226


Source    Test Once    Objects: Call by ref Versus Call by value
Call by Ref x 2.29 faster 0.0024 Call by Value 0.0055

In PHP5b4, object passing has been optimised, so there is little difference between 'call by reference' and 'call by value'. However it appears that some optimisation work still has to be done, as passing objects in PHP5b4 is slower than in PHP4.

Source    Test Once    Arrays: Call by ref Versus Call by value
Call by Ref x10.10 faster 0.0021 Call by Value 0.0212


Source    Test Once    Objects: Call by ref Versus Call by value
Call by Ref 0.0061 Call by Value x 1.02 faster 0.0060

The overhead of passing in arrays remains high in PHP 5. This suggests that in PHP 5 we should continue to call arrays by reference, or encapsulate all arrays we are passing in a wrapper object class such as:

class phparray
{
var myarray = array();
}

There are several other benchmarks in this section. It appears to make no difference whether we pass strings by reference or by value. We also tested long function names and short function names. Calling functions with short function names were again faster. There are two common ways to return multiple values from a function. You could pass reference parameters into the function and update those parameters, or return using an array:

return $array;

It seems using an array is faster.

XML Suite
These tests measure the time taken to parse an RSS newsfeed for all title tags. I tested using regular expressions, explode with substr, DOM, XPath, and SAX. I was surprised to find the following regular expression gave the best performance:

preg_match_all(
'/<title>([^<]*)/',
$XML,
$titles_array)

SAX came last in performance. This was another surprise as people have always said that SAX gives better performance than DOM, because DOM recreates the XML structure in memory, while SAX does not. The reason why regular expressions are fastest is because regular expressions do not have any knowledge of XML. This means that tag validation is neither required nor performed. The other surprise, that DOM is faster than SAX, can be explained like this - a large percentage of the time in SAX processing is spent in callbacks to slower PHP code. In contrast, DOM generates the XML structures in very fast C code with no PHP callbacks required. So the conventional wisdom that SAX is faster than DOM is true when everything is written in C, but not in hybrid environments such as PHP with C extensions. Following are the Windows results for PHP4:

Source    Test Once    Parsing XML - perl regex Versus explode
preg_match_all 0.1004 explode x 1.13 faster 0.0888


Source    Test Once    Parsing XML - explode Versus DOM XML
explode x14.56 faster 0.0949 dom xml 1.3815


Source    Test Once    Parsing XML - explode Versus XPath
explode x18.26 faster 0.0908 xpath 1.6580


Source    Test Once    Parsing XML - explode Versus XML parser (SAX)
explode x91.78 faster 0.0881 xml parser (SAX) 8.0856

Algorithms Suite
This suite was recently added to measure various algorithms. One common task is encoding binary data for XML, or data storage. The na? way is to use rawurlencode(). Turns out that it's dead slow. Base64_encode() appears to be the fastest. PHP has multiple functions for generating hashes. This is useful for generating unique passwords, or generating a security checksum for verification purposes. We compared crc32 and md5. crc32 was faster, though md5 is known to generate more unique hashes. And lastly we test storing configuration parameters in an INI file or in a PHP file as an associative array. We found that storing in an INI file and using parse_ini_file() is faster than parsing a PHP file.

Benchmark Suite System Design
The BS System is designed as a set of independent test suites. Each suite is stored in its own sub-directory and each test is a separate PHP script file. You don't need to perform any special setup to add new tests. Create a new sub-directory, and the BS System will auto-detect the new test suite. Any .php script files found in a suite directory will be executed in alphabetical order. Each script file should have one or two benchmark functions defined, bench1() and bench2(). Each function should return a value that can be used to verify that the function actually worked. There is also some required metadata embedded as a comment in the source code. This is read by DoScanDir( ) in bench.php. The metadata format is (in one line):

//~~ Title to display,
// description of bench1(),
// description of bench2(),
// # iterations

Note that #iterations is optional, and will default to 1000 if not defined. Check out the sample test script in Listing 1.

Listing 1

<?php

include_once("../init.inc.php");

// METADATA THAT IS READ BY ../bench.php:
//~~ Reading CONSTANTS versus Global Variables, Constants, Globals, 1000

//================================================ INIT

define('CONSTANT',1);
$CONSTANT = 1;

//================================================ TEST CODE

function bench1()
{
    return CONSTANT+CONSTANT+CONSTANT;
}

function bench2()
{
global $CONSTANT;

    return $CONSTANT+$CONSTANT+$CONSTANT;
}

//================================================ BENCHMARK!

include_once("../bench.inc.php");
?>

You can also cache the results by setting $CACHE=1 in config.inc.php. The results are cached in the _cache directory. You can save the contents of this directory for a permanent record. John runs the popular php.weblogs.com Web site, and is the lead developer of the phplens app server.

Links and Literature:

© 2004 Software & Support Verlag GmbH. Reproduction has to be permitted by the publisher. Questions?