experimentalworks » PHP

php-trunk macport

dsp — Mon, 24 May 2010 15:48:39 +0000

macports is a widely used ports system for Mac OS. It’s repository contains hundreds of application that can be compiled and installed. The repository contains php 5.3. So if you want to run PHP from subversion you still have to compile it yourself and install it yourself outside your managed ports environment. I created a rather simple Portfile to build it from PHP’s trunk.

To use the php-trunk portsfile. Just extract the tarball into a directory and…
$ tar xzvf php-trunk-port.tar.gz $ cd php-trunk-port $ echo "file://`pwd`" > /opt/local/etc/macports/sources.conf $ echo "file://`pwd`" >> /opt/local/etc/macports/sources.conf $ port install php-trunk

And the best thing about the port: It compiles PHP with dtrace support :).

Please note the Portfile is very simple and is not tested with the additional modules provided by macports. So use it at your own risk and enhance it.

DOWNLOAD

UPDATE: as philip noted, it should be >> instead of >.

Writing a simple PHP sourcecode buildscript in Scala

dsp — Sat, 09 Jan 2010 22:48:22 +0000

Scala is a fascinating language. Running on the Java VM, Scala offers a powerful mixture from both the imperative Java world and functional programming including modern techniques like Actors. Personally I prefer to not just learn programming languages, but also try them out while reading through the book.

As I wanted to have nice a build system for my PHP subversion checkout, I used this need as a project to start coding Scala. So what do I exactly need? I want to build multiple versions of PHP from the same branch without checking out the code twice. I also want to configure these builds somewhere without always typing in the parameter list or so. For further versions I want to be able to configure these in a file that can easily be distributed to other machines.

I set down and wrote a parser for a configuration file that can configured build targets which is then build by
the program. The configuration file I used is specialized for this purpose, which is why I didn’t used something like ant or so. The result is called bauaffe-3.0.0a1.jar.

I’ll just show a few things done in the project, but mainly focus on what the nice script can do. Further blog posts will be about the actual implementation.

The configuration looks like this

$ cat ~/.buildmaker
begin default configuration
    define source "/Users/dsp/dev/c/php-src"
    define build "/Users/dsp/dev/c/php-src/build"
    define defaults as
        with "iconv=/opt/local"
    build trunk as
        "php60" using defaults
        "php60-debug" using defaults
            enable "debug"
    build branch "PHP_5_3" as
        "php53" using defaults
            environment PHP_AUTOCONF="autoconf213"
        "php53-debug" using defaults
            enable "debug"
            environment PHP_AUTOCONF="autoconf213"

.
Proper indention is not necessary (as e.g in python).

You might want to think that parsing the configuration file can be difficult. Well, if you use C you would use YACC, if you use PHP, I don’t know what you would have done, but Scala is made to create this kind of Domain Specific Languages (for my the config is a DSL). You can easily transform a EBNF directly to scala code using the JavaTokenParsers provided by the Scala Library. As an example this it the statement that parses the first line:



    def begin : Parser[Configuration] =

        "begin" ~ ("default" | stringLiteral) ~ "configuration" ~ rep(define | build) ^^ {

        case "begin"~name~"configuration"~confs => new Configuration(name, confs)

    }

which is directly taken from the BNF:



      config ::= "begin" ( "default" | string ) "configuration" ( define | build )*

Did I mention that the actual parser is 170 lines of code with usual indention and formatting?

Configuration
The configuration file is searched in ~/.buildmaker, or if ~/.builmaker doesn’t exists, buildmaker.conf in the current directory. How do you configure the tool? First of all you can specify a configuration. It is usually called “default”. It is not yet supported to name it differently, although the parser is able to parse it. In further versions multiple configurations per file are allowed.

Variables
Variables are set using the define syntax. At the moment you can set the build and source variable as well the defaults variable, which is usually a block of statements that can be used in the branch configurations.

Branches
A branch is configured using the build syntax. You first have to specify which branch to build. Every branch can then configured to have build target with a given set of options. Branch options are:

with string: Builds the target with the given extension
enable string: Builds the target with the given extension
environment string=string: Builds the target with environment variable

. You can specify using defaults which will cause the runner to use the options specified in the defaults define.

At the moment the parser will not do a good job in notifying you what you are allowed to do and what not, although pure parse error will be emitted. You can also not set any other variable than the described once.

Building
Calling

$ java -jar bauaffe-3.0.0a1.jar list
TARGET                         LAST BUILD
php60                          None
php60-debug                    None
php53                          Sat Jan 09 16:55:12 CET 2010
php53-debug                    Sat Jan 09 16:59:37 CET 2010

gives you a list of parsed targets and their last build date. You can build a target using

$ java -jar bauaffe-3.0.0a1.jar

or build all using

$ java -jar bauaffe-3.0.0a1.jar all

Please notice that the current version requires that you now what you are doing. You might miss some error messages or find them not useful. I’ll change this before the first release, if I’ll do a final version of it. I hope you like the little tool.

Download It!

Scala (pronounced /ˈskɑːlə, ˈskeɪlə/) is a multi-paradigm programming language designed to integrate features of object-oriented programming and functional programming.[1] The name Scala stands for “scalable language”, signifying that it is designed to grow with the demands of its users.

Calling Conventions – when you need to know C to understand PHP

dsp — Mon, 06 Jul 2009 16:26:59 +0000

I think most of the people using PHP wonder from time to time about particular behavior of the language. That’s pretty much the same case with every language. Pythoneers have their wtf moments, Ruby programmers have their wtf moments and C programmers tend to live in a whole wtf universe. But lately I stumbled over a nice one. It looked like a bug in PHP, but turns out to be an interesting, curious, part of the C-language. Imagine the following PHP code sample and note that $a and $b are not defined (yeah I know, it’s bad coding style..blabla..):
var_dump($a + $b); ?>
What is the expected result with error_reporting set to E_ALL?
PHP Notice: Undefined variable: b in /var/foo/bla on line 1 PHP Notice: Undefined variable: a in /var/foo/bla on line 1.
Are you sure? I’m not. On x86 hardware b is fetched before a is fetched and therefore the executor detects that b is not set first. But wait. Let’s test this on a SPARC machine:
PHP Notice: Undefined variable: a in /var/foo/bla on line 1 PHP Notice: Undefined variable: b in /var/foo/bla on line 1.
What? It evaluates it in the reversed oder? What is happening? So I spend a few minutes with my lovely debugger and it turns out that this is what happens in the engine (I use pseudo code here):

result ZEND_ADD_OPCODE()
{
   return add_function(get_op1(), get_op2());
}

Voila, that’s the problem. On SPARC get_op1() is executed before get_op2(), while it’s the other way round on x86. As get_opX() detects if a variable exists, the error messages appear in reversed oder. I did a little bit research (thank you SunCC Team for your answer!) and it turned out, that C99 doesn’t define the way function calls in parameter lists are executed. Therefore, every system and compiler is free to use it’s own ordering mechanism. My current plan: Write a compiler that does this by random(). The fix is trivial:

result ZEND_ADD_OPCODE()
{
  op2 = get_op2();
  return add_function(get_op1(), op2);
}

. It’s a lovely curiosity.

DTracing PHP

dsp — Sat, 20 Dec 2008 01:22:09 +0000

Dtrace is an extremely flexible and powerful tool to trace and debug applications. I recently dtraced PHP a little bit, so here is a tutorial how to start dtracing PHP.
Prequisites
You need a dtrace capable operating system like Solaris 10, OpenSolaris, Mac OS X 10.5 or FreeBSD 7. I use dtrace on OpenSolaris 2008.11. You also need at least dtrace privileges or need to be root to be able to trace a program. I used PHP 5.3.0alpha3 in my tests, but it actually doesn’t matter. As PHP doesn’t provide DTrace probes itself we have to trace the underlying C-functions, therefore we need to know the Zend Engine and PHP’s internals.

An Introduction into DTrace
Dtrace is really powerful and trying to do an introduction to all it’s features is just not possible. Therefore I will just focus on the basics, that are needed to get our stuff working. The basic idea behind Dtrace is that the kernel and userland programs fire probes on a specific location in the kernel or the userland program. As the probe is just fired if DTrace is running and trying to catch those markers, the probes itself don’t cost CPU power at all (in fact they are NOP’s). But if we trace them, we are able to see those probes and they are able to pass some data. To get way more complex information out of those probes, DTrace has an builtin scripting language that catches those probes and process them.

Dtrace program structure
Every probe in DTrace is describe by a 4 tuple. The first one is the so-called provider. The second, the probemodule, the third is the probefunc and the last one is the probename. The parts are separated by a semicolon, therefore every probe is defined by:

:::

In our example, the provider will usually the pid provider that gives us probes for a given PID (the PHP controller we want to trace). The probemodule will be php and the probefunc will be the function name of the underlying C-function. The name can actually be everything but it is best-practice to have a probe at the entry of a function, called entry and returnjust before the function returns. A DTrace program have an probe identifier that matches a probe. Every identifier has a body that can contain initialization of variables or accumulation of data using a table. Also, every identifier can have a condition that is evaluated before the body is run. So we have the following structure:

:::
/ /
{

}

As an example, we trace gc_collect_cylce function:

pid$target:php:gc_collect_cycle_function:entry
{
trace(probename);
}

Please notice that DTrace features a lot predefined variables, such as probename. Please refer to Sun’s Dtrace Guide to get a list of predefined variables.

PHP probes
~~As PHP doesn’t provide own probes we have to use the underlying C-functions.~~ PHP itself doesn’t provide own probes. There is ext/dtrace written by Wez Furlong which gives PHP some probes, but as we want to get a little bit deeper, we have to use the underlying C-functions. Userland programs are usually traced by the so-called pid-provider. It’s used to get all probes by a program with a given PID. Unlike userland programs, the kernel doesn’t have a pid, therefore kernel probes do not have a PID and can be identified easily be an unique name. As we sometimes do not know the PID of the php process that we want to trace, there is a variable called $target. If yo use it, $target will be automatically filled with the PID that was created when dtrace started the program. If I start a program with dtrace -s test.d -c ‘php test.php’ the programm php test.php will be executed and the created PID will be assigned to the variable $target. DTrace also have an option just to display the available probes: Try dtrace -ln ‘pid$target:php::entry’ -c ‘php test.php’ to get all function entries called by PHP. You might from time to time a get a message that not enough memory was available to trace. In those cases you just tried to capture too much probes. Therefore, I used the specific :::entry notation to just get function entries, which worked fine on my machine. So let’s take look at some interesting probes. If we just try to display all probes related to PHP with dtrace -ln ‘pid$target:php::entry’ -c ‘php test.php’.
The output might start with something like:

ID PROVIDER MODULE 69608 pid2696 php 69609 pid2696 php 69610 pid2696 php 69611 pid2696 php 69612 pid2696 php 69613 pid2696 php 69614 pid2696 php 69615 pid2696 php 69616 pid2696 php 69617 pid2696 php 69618 pid2696 php 69619 pid2696 php 69620 pid2696 php 69621 pid2696 php 69622 pid2696 php 69623 pid2696 php 69624 pid2696 php 69625 pid2696 php 69626 pid2696 php 69627 pid2696 php FUNCTION NAME
_start entry
__fsr entry
_free_ereg_cache entry
zm_startup_ereg entry
zm_shutdown_ereg entry
zm_info_ereg entry
zif_ereg entry
zif_eregi entry
php_ereg_replace entry
zif_ereg_replace entry
zif_eregi_replace entry
zif_split entry
zif_spliti entry
zif_sql_regcase entry
php_regcomp entry
p_ere entry
p_ere_exp entry
p_str entry
p_bre entry
p_simp_re entry

First example
So our first example will be that we want to trace the compilation time. The compile function is called compile_file in PHP. Therefore our probe identifier would be pid$target:php:compile_file. The PID provider will just get the probes for a specific probe, while the php probemodule will focus on all PHP related functions. We then select the compile_file function. Provided by the kernel, there are two probenames, calle entry and . Entry is called when the function is entered and return, when the function returns. So we need to trace pid$target:php:compile_time:entry and pid$target:php:compile_time:return. We know just have to get information how long the compilation takes. For that purpose DTrace defines a variable called timestamp that is accessible within the body of an DTrace identifier which holds the milliseconds from the beginning of a program. So know we catch our entry probe and set the time to a local value:

pid$target:php:compile_file:entry
{
self->compile_start = timestamp;
}

Note that the self-> notation is used to get a per-thread variable. Therefore every thread would have it’s own value. It’s not particularly necessary for our PHP based example, but it is best-practice to use thread-local variables whenever possible.

We just need our return probe that calculates the real offset:

pid$target:php:compile_file:return
{
printf(“Compile time: %dn”, timestamp – self->compile_start);
}

This will output our actual compile time. Just put both identifiers and their bodies into ‘compiletime.d’ and start dtrace with dtrace -q -s compiletime.d -c ‘php test.php’ and it will display you the compilation time of test.php. Please notice that both self->compile_start and timestamp are integers. Dtrace in fact doesn’t have the notion of a float primitive, so you would not be able to output a float using printf.

The output might be similar to:

$ dtrace -q -s compiletime.d -c ‘php test.php’
Compile Time: 99604

Advcanced Example
Now let’s get a little bit deeper into the engine. We now want to trace when the garbage collector hit’s in and want to see the amount of freed refs. The garbage collector uses the function gc_collect_cycles.

pid$target:php:gc_collect_cycles:return
{
printf(“%d refs freed”, arg1);
}

Please notice the special arg1 variable. This always holds the return value in “return” probenames. In “entry”, arg0…argN will hold the function parameters.

For more detailed examples, see my next blog entry, which will contain more sophisticated DTrace scripts.

Compiling PHP under OpenSolaris

dsp — Tue, 30 Sep 2008 01:17:37 +0000

As I switched my main system recently from Linux to OpenSolaris I compiled PHP. Quiet obvious things are a little bit different on Solaris. The usual ./buildconf && ./configure && make install doesn’t work anymore. The good news: It’s not much harder. Let’s start to get the prequisites:

~$ su
~# pkg install SUNWgm4 SUNWaconf SUNWgmake SUNWgcc SUNWbison

Finally we have to get the latest re2c sources. Re2c is needed to regenerate PHPs parsers but not yet available in OpenSolaris official package repository. So go to re2c at sourceforge and download the most recent sources.

~$ gtar xzvf re2c-0.13.5.tar.gz
~$ cd re2c-0.13.5/
~$ ./configure
~$ gmake

The other necessary GNU tools should be installed, otherwise try to install SUNWgnu-coreutils. So let’s get our CVS snapshot and compile it:

~$ cd php-src-5.3/
~$ MAKE=gmake ./buildconf
~$ ./configure RE2C=”~/dev/c/re2c-0.13.5/re2c”
~$ gmake
~$ su
~# gmake install

As we have to use GNUs make and not Solaris make implementation we have to set the MAKE environment variable used by buildconf. Since re2c is not in our path (we didn’t installed it), we pass the location of the re2c executable to the configure script with the RE2C env variable. After successfully executing gmake install, we have our PHP 5.3 installed.

~$ php -v
PHP 5.3.0alpha3-dev (cli) (built: Sep 30 2008 02:37:10)
Copyright (c) 1997-2008 The PHP Group
Zend Engine v2.3.0, Copyright (c) 1998-2008 Zend Technologies

Please notice that PHP’s default configuration is to install the binaries in /usr/local/bin. This directory is usually not in $PATH, so update your PATH.

UPDATE
As pointed out in the OpenSolaris Forums GNU make is just required for the buildconf step, so you should be able to use Solaris make implementation if you don’t compile from CVS sources.

Extension development on windows

dsp — Mon, 04 Aug 2008 10:32:00 +0000

When it comes to choose a operating system to run PHP applications often Linux is the choice. PHP, it’s buildsystem and it’s extensions are developed and optimized to run under Linux (and Unices). There are several reasons that Linux is the better supported operating system. Maybe due to historical reasons, maybe most of the core developers work und Linux (rumours were afloat that this is not quite true) or maybe just because the extensions finally run on a Linux server, so therefore it only counts if it runs under the target system.

This is actually the reason to not even try to compile my ktaglib extensions under Windows. Well but this approach is doomed to fail as most of the developers out there use Windows as their operating system. Therefore they are not able to run the extension and might not be able to ingerate your extension into their application. So actually I ended up booting up windows again (well after searching for the harddisk that contains a running windows) and try to compile it. Otherwise my fellow collegues wouldn’t be able to fix my buggy PHP code.

I remember compiling extensions under Windows a year ago, which actually doesn’t seems to be very easy for me. Since then a lot changed. Ongoing pecl2 efforts from Pierre with great help from Elizabeth and Rob now makes compiling on Windows without deeper knowledge about the Windows buildsystem easy. They carry together various notes about the build dependencies in the official php wiki. Furthermore Elizabeth recorded a video showing how to compile PHP on windows in a few minutes. So I actually ended up going to the pecl2.php.net page and just get the necessary libraries, put them into the right directory and compile PHP the same way as under Linux.

This helped to actually get ktaglib running under Windows. Just take the taglib windows port from the taglib website and compile your windows ktaglib using the PHP windows build system. Maybe the new scripts from the php-internals-win repository, that Pierre recently commited, help you to get you way through. It’s really amazing to concentrate on the work and not trying to get a buildsystem working.

ktaglib is a PHP binding for KDE’s taglib that helps you to read id3 tags and audio information from MP3s or OGGs. The current version is 0.0.1a1 and can be downloaded at http://pecl.php.net/package/KTaglib

getopt and longopts in PHP 5.3+

dsp — Wed, 03 Oct 2007 23:18:01 +0000

In the upcoming PHP 5.3+, the getopt() function will finally be available on Windows systems. Furthermore getopt() will support longopts on every system. To support this feature we moved the generic getopt function used by the PHP cli/cgi interpreter itself to main/ and recode getopt() to use this function.

Thanks for Jani to review and commit my patch.