Warning this is a potential boring blog entry. Apprx writting time was about 5min

I once tried writing applications for the iPhone and started to like the concept of Obj-C while still I hate the syntax and I don’t like the API that much. Android makes things simple for me, and I guess for most people knowing a little bit java. It implements big parts of the Java standard (although it doesn’t claim to be standard compatible) which makes it easy to start writing apps if you know the libraries a little bit. Android also includes the powerful HTTP libraries of the Apache project as well as Json and XML parsers.

Having said this, the most important factor why I started to love my phone and the Android system: I can easily deploy apps on my phone, I debug it on my phone, I have nearly the complete power of the java framework and I totaly enjoy seeing my first app starting on the actual phone after 1 hour of coding. The app itself is fairly simple. It’s a basic search interface phpMyFAQs upcoming Rest API.
So you can search an existing phpMyFAQ installation for a search term and display the result. Although the app needs phpMyFAQ 2.6.0-alpha it’s already available in the Android Market. So feel free to check it out. You can test it with the URL: http://faq.phpmyfaq.de/. The source can be found at http://github.com/thorsten/pmfMobileKit. Expect more updates to the app soon.

Inhaltlich ist das CRE eher technisch aufgestellt und befasst sich mit allem was den Nerd erfreut, sozusagen die Cub-Mate des audiophilen Tekki. CRE ist also gut für die Geek-Gesundheit, liefert einen Blick über den Tellerrand und, hepp, befasst sich sogar mit Soziologie und Politik. Und manchmal werden sogar langjährige Wünsche erfüllt.

Absolute Highlights sind natürlich Folgen wie die Bayrische Hackerpost, Internet Memes und der Mifare-Hack. So ist es kaum verwunderlich, dass man nach späterstens zwei Wochen CRE freie Zeit anfängt unruhig den CRE Blog anzusurfen und den Podcast Feed zu abonnieren. Leider fehlt immer noch was zum Thema PHP, aber das ist ja per se uncool, deswegen wird das wohl auch nichts mehr. Schade eigentlich, vielleicht gibts ja nochmal was dediziertes zum Thema Flamewars, da kann man PHP dann immer gerne einstreuen. Überhaupt gibt CRE immer Anlass zur Diskussion, meistens weil ein Detail in irgendeinem Nebensatz nicht stimmte oder allgemein der Gast inkompetent war, aber eh klar: man selbst ist immer der bessere Experte (vorallem wenn es um DVCS geht, wo’s vielleicht sogar stimmen würde *rant*). Aber auch “ranten” gehört zur Informatikerausbildung. Also immer brav CRE hören.

Was man am CRE verbessern kann?:
Ich finde die Idee mit einem Photo pro Podcast eigentlich ganz gut. Mehr fällt mir auch nicht ein.

Und jetzt weiter in der von Tim initiierten Blogkette:
Zurück zu Nummer 12 in der Blogkette.
Weiter zu Nummer 14 in der Blogkette.

und jetzt weitermachen!

/usr/include/bits/stdio2.h:98: undefined reference to `__builtin_va_arg_pack’

At that moment just one thing came into my mind: WTF! Well but as usually, the web knows the answer, as long as you know how to find it.

So let’s go back and start over how to setup distcc on both machines:

ubuntu $ sudo apt-get install distcc
gentoo $ emerge distcc
gentoo $ rc-update add distccd default
gentoo $ vi /etc/conf.d/distccd
and set DISTCCD_OPTS=”${DISTCCD_OPTS} –allow 192.168.1.0/24″
save the file and quit vi.
gentoo $ /etc/init.d/distccd start

Get PHP and extract it on the ubuntu machine:

ubuntu $ wget “http://www.php.net/get/php-5.2.6.tar.bz2/from/www.php.net/mirror”
ubuntu $ tar xjvf php-5.2.6.tar.bz2
ubuntu $ cd php-5.2.6/

So know here is the trick. The undefined reference error occurs as gcc(1) on ubuntu compiles with -fstack-protector, which is not enabled by default on Gentoo. We have to set it during configure so that the option is passed to the gentoo machine and the compilation fits:

ubuntu $ ./configure CC=”distcc” CFLAGS=”-fstack-protector”
ubuntu $ export DISTCC_HOSTS=”localhost gentoo”
ubuntu $ make -j6

And it works! Compilation is done!

and now the boring stats:

make -j3 without distcc
198,27s user 43,41s system 184% cpu 2:11,10 total
make -j6 with distcc
62,69s user 38,02s system 186% cpu 54,044 total

When lighttpd got a request it can call a Lua script before the fcgi take over the request. The lua script is able to modify the complete request handling. Jo tried to use this abillity to make a cache decision in lua and therefore save the PHP + FCGI overhead by serving the cache if available. As this approach usually makes sense on high traffic sites, people want to use a distributed caching system like memcached. The cache can be run on a complete other machine and requested over network. To interact with memcached Jo used the Memcache.lua bindings, which are quite similar to other language bindings like the PHP bindings. Unfortunatly when calling Memcached.Connect from lighty, it crashed silently. If you tried to call memcached from the standalone lua interpreter, everything worked fine. So what happend?In fact not the memcache bindings caused the problems, but the underlaying luasockets library. I discovered that luasockets fails while initializing the tcp socket and setting up a buffer with the information to send through the socket. First I thought it was a problem with the way lighty handles sockets. Maybe this could confuse luasockets. Apparently it doesnot. The socket gets perfectly allocated by the system and returned.

I was pretty confused as I tried to look at the exact point where the execution stopped. The buffer_init method which was called returned a invalid pointer, even the function itself was so clear, that I cannot break the library. Everytime I tried to step into the mysterious buffer_init method of luasocket gdb seems to jump to a wrong line and not to the buffer_init method. Deciding to renew my source directories in gdb actually made my day. Suddently gdb switched into lighty’s buffer_init script.

Pretty obvious: lighttpd also has a buffer_init method. When loading the luasocket library as a shared module, it’s buffer_init method address gets overwritten by lighty’s buffer_init. Therefore the luasocket always used light’s function which initialized the buffer in an unexpected way and caused the luasocket to get a null pointer instead of a correct initialized value. As luasocket doesnot check this, it fails when trying to access the pointer and segfaults (together with lighty).

So the actual fix is to prefix all of luasockets exported functions as well as their references and recompile.

Well sadly I still have some various crashes based on race conditions during startup. I didnot have the time yet to fix that, but next time I take
hands on vcap, that will be fixed.

What is vcap?
Vcap captures ethernet packages like tcpdump or wireshark and displays statistik about them using modern visualization techniques. At the moment the statstics are drawn using a sunburst algorithm. It aims to give you a much fast overview about the network statstiks than tcpdump or netstat or something similiar does. It’s GTK based. Vcap is not yet stable and it is known that it has some issues that causes vcap to fault. But work goes on.

Our first approach was using git-cvsimport which ships with the GIT distribution. git-cvsimport is based on cvsps which parses the output of rlogs on a current working copy and receives all revision for all files in the working copy and imports them into a git repository. It turns out that git-cvsimport works quite well for small projects, but for large and complex CVS repositories like the PHP one, git-cvsimport fails. It imports most of the files but from time to time it messes up with the import of revisions or even complete branches. That leads to a broken repository that was not able to compile anymore. This behaviour seems to be a result of the cvsps import mechanism based on rlogs which doesn’t have enough information to make all mappings of files to branchnes.

Therefore we tried parsecvs. Instead of git-cvsimport, parsecvs works on ,v RCS files from the CVSROOT. It parses them and imports the revisions into GIT. As php.net provides an rsync mirror to get the CVSROOT sources (thanks to Derick for adding ZendEngine2 to the rsyncs), we were able to import the PHP repository into GIT. But we still run into troubles. The PHP CVS obviously has a branch name dev/. That branchname is passed to git-update-ref by parsecvs. This git command is based on git refspecs which uses slashes to separate names. As the slash in dev/ was not escaped right, we needed to patch parsecvs ourselfs to escape (in fact use ~ for /) that character. Besides that fix, which was done by me, Johannes fixes some other issues in our own parsecvs branch.

Thanks to our parsecvs we finally got a working GIT repository imported from CVS. It was just about 2,2 Gb big. A mail from the git mailinglist concerning the GCC GIT repository leads us to the right solution to get that repository smaller by recalculating the revision deltas using git-repack, which tooks just about 3 hours. That attemp leads to a perfect and small 94MB repository.

This inofficial GIT repository is updated from the official PHP CVS 2 times per day. It is served by Johannes Schlüter, who also provides imports for ZendEngine2 and TSRM which are needed to compile PHP. You find a mirror of these repositories on my GIT site.

[LINK] The inofficial GIT PHP repository.
[LINK] The inofficial GIT TSRM repository.
[LINK] The inofficial GIT ZendEngine2 repository.

Feel free to pull the repositories and send patches to us using GIT. Feel free to drop me a comment.