Tag Archive for 'testfest'

PHP TestFest 2008

PHP.net has posted a nice overview of the PHP TestFest 2008 initiative, which aims to increase the PHP source code coverage through PHPT tests. Worldwide, the TestFest was carried out with different user groups, and of course the PHPBelgium and phpGG were also present. We arranged a meetup in Roosendaal earlier this year and with a group of 10 people, we managed to write about 40 tests! All efforts combined, there was about 10 % increase in code coverage for PHP, which was great to hear!

Actually, I found it so much fun and rewarding to help PHP as a whole in this new way, I decided to continue contributing tests to the submission system. You can imagine I was quite excited when the PHP-QA team actually asked me if I was interested in commit access to the PHP source tree! It’s a great way to give back to the community, and apparently I got mentioned for this in the php.net article, which of course made me somewhat proud ;-)

It was very cool to see the collaborative work around the world being rewarded with our favourite pets, and no less than 4 people from Belgium and the Netherlands (Michelangelo van Dam, Stefaan Koopmanschap, Rein Velt, Marc Veldman) were given an elePHPhant.

Actually I think initiatives like the PHP TestFest make this thriving and exciting community what it is right now, and it’s because of that I very much like the Emphpower initiative from Lucas Smith. To quote the mission statement: “emPHPower is a mediator and catalyst that empowers members of the community to follow their own ideas”. For example, things like the PHP TestFest could fall under the organisation of Emphpower.

I’m pretty much sure I’ll organise the next year PHP TestFest for Belgium (and the Netherlands), under the PHPBelgium / phpGG umbrella of course ;-) Testing is one of many great ways to engage in the PHP community, and the only thing you need to know is how to write PHP code. After all, most of you who are reading this probably know how :-) More information can be found on the PHP-QA website.

PHP TestFest 2008 on PHP.net” screenshot on flickr.

PHPT (testfest) writing resources

The PHP TestFest month is almost over and we can already probably say it was quite a success. Of course it would be great if some of the test writers continue to occasionally write a new test or two, and for this occasion, I thought it would be handy to have a resource listing of presentations, websites and pdf’s that cover PHPT testing. Although I believe this is a decent list of resources I’ve actually used myself, I’m probably missing some, so this will be a work in progress and updated once in a while. :-)

First, some of the official PHP resources on PHPT testing (pretty much all of qa.php.net is relevant here):
http://qa.php.net/write-test.php
http://qa.php.net/running-tests.php
http://qa.php.net/tested-methods.php
http://qa.php.net/testfest.php
http://qa.php.net/phpt_details.php
http://wiki.php.net/qa/testfest

GCC code coverage page for PHP, which shows test code coverage for the most popular CVS branches. Some graphs and other compiling, testing and analysing information is also available:
http://gcov.php.net

This is the place you’ll need to go to acually submit tests if you don’t have CVS commit karma yet (it’s good for overall testing statistics too):
http://testfest.php.net

Next is a blog post on what Josie Messa has learned while writing tests. Interesting article which shows a lot of real code examples and issues that she dealt with while writing PHPT tests:
http://jmessa.blogspot.com/2008/05/lessons-learned-in-phpt-writing.html

Slides by Sebastian Bergmann, useful to prepare yourself for writing tests. This presentation tells about what PHPT testing is all about, how to properly write tests, and how you should prepare for testfest:
http://sebastian-bergmann.de/archives/777-PHP-Test-Fest-Slides.html

This is an article I wrote, explaining a way to set up Mac OS X for writing PHPT tests. This doesn’t use the built in PHP distribution but instead prepares the OS for manually compiling and running PHP from CVS, with code coverage and all the goodies:
http://felix.phpbelgium.be/blog/2008/05/22/writing-phpt-tests-on-your-mac/

Presentation by Marcus Börger with pretty extensive overview on testing (and PHPT testing in specific). This is one of the most complete presentations on the subject I could find:
http://somabo.de/talks/200703_montreal_need_for_testing.pdf

The blog of Zoe Slattery, which has 2 posts (and much more to come, undoubtly!) about testing PHP and PHP TestFest:
http://zoomsplatter.blogspot.com/

This is the official website of the PHPT testing framework:
http://phpt.info/

Of course you’re best off grabbing a CVS checkout of the PHP_5_3 (or 6_dev) source tree:
http://www.php.net/anoncvs.php

In the CVS checkout of the PHP source code, there is a interesting README file called README.TESTING. It provides useful information on how to test, testing conventions, how to handle failed tests and more:
http://cvs.php.net/viewvc.cgi/php-src/README.TESTING?revision=HEAD&view=markup

If you have questions regarding writing PHPT tests, there are several ways to go. First off, you can ask away at the IRC channel #phptestfest on irc.freenode.net. For any kind of QA or testing related question, you can also mail php-qa@lists.php.net mailing list, and subscribe to the mailing list on http://www.php.net/mailing-lists.php.

Writing PHPT tests on your mac

This is a short tutorial on how to set up a Mac with Mac OS X to prepare for writing PHPT tests. This preparation includes installing some extra software that enables you to checkout and compile the PHP_5_3 branch of the PHP source without problems. Furthermore we’ll install some tools that make it possible to get an easy overview of untested PHP code with code coverage reports.

This overview assumes you’re working on the latest version of Mac OS X, being 10.5 (Leopard). Some terminal experience also comes in handy!

1. Installation of XCode

Apple’s XCode is used to create a development environment on your mac. For example, the GCC compiler, automake and other build tools are installed with XCode. Notice: you do need XCode version 3 or higher. The files needed for the installation can be found on the Mac OS X installation DVD or on the Apple Developer Connection website (http://developer.apple.com)

2. Installation of MacPorts

MacPorts is a package manager that will ease the installation, configuration and updating of open-source software on your mac. We’ll use MacPorts to install some tools that are needed in our development environment. You can find the MacPorts installation files on http://www.macports.org. By default, the MacPorts binaries are installed in /opt/local/bin. To make working with the terminal easier, we’ll add the directory to our PATH variable. Let’s open Apple’s Terminal.app and type:

# echo 'export PATH=/opt/local/bin:$PATH' >> ~/.profile
# source ~/.profile

Trough MacPorts, we’ll install a lexer used by PHP (re2c) and a tool to download files (wget):

# sudo port install re2c
# sudo port install wget

3. Lcov installation

Next up, we’ll install lcov. Lcov is a graphical front-end for gcov, a code coverate testing tool for the GCC compiler. Lcov isn’t available in the MacPorts repository, so we’ll need to install this manually:

# sudo mkdir -p /usr/local/src; cd /usr/local/src
# sudo wget http://downloads.sourceforge.net/ltp/lcov-1.6.tar.gz
# sudo tar -xzvf lcov-1.6.tar.gz
# cd lcov-1.6

Due to a non-compatible ‘install’ binary on Mac OS X, the installation can’t be completed with the -D switch. This has no implications on the actual resulting installation, but it does need to be changed to suppress the error messages:

# sudo nano /usr/local/src/lcov-1.6/bin/install.sh

Go to line 34 (install -D $SOURCE $TARGET) and remove the -D switch. Afterwards you can install lcov with the following command:

# sudo make install

4. Installing the right flex version

Now we need to install a alternate flex version. By default, Mac OS X has the flex lexical parser installed, but the installed version is not compatible with PHP. Thus we’ll install an older flex on an alternate location, which then can be used for compiling PHP:

# cd /usr/local/src
# sudo wget http://dfn.dl.sourceforge.net/sourceforge/flex/flex-2.5.4a.tar.gz
# sudo tar -xzvf flex-2.5.4a.tar.gz
# cd flex-2.5.4
# ./configure && make
# sudo make install

5. Installing the right autoconf version

If you compile PHP, you’re advised to use autoconf version 2.13. Since Mac OS X comes with autoconf version 2.61 by default, we’ll install an older autoconf version, which again can be used when we’re compiling PHP:

# cd /usr/local/src
# sudo wget http://ftp.gnu.org/gnu/autoconf/autoconf-2.13.tar.gz
# sudo tar -xzvf autoconf-2.13.tar.gz
# cd autoconf-2.13
# ./configure && make
# sudo make install

Let’s change our PATH variable. That way, the right lcov, flex and autoconf binaries will be found by PHP:

# echo 'export PATH=/usr/local/bin:/opt/local/bin:$PATH' >> ~/.profile
# source ~/.profile

6. Downloading the PHP source

We create a directory to put all our sources and other related files:

# mkdir ~/phptest; cd ~/phptest/

Now we can check out the PHP source code using anonymous SVN:

# svn checkout http://svn.php.net/repository/php/php-src/branches/PHP_5_3 php-src-5.3 
# cd ~/phptest/php-src-5.3
# ./buildconf

7. Compiling and testing of PHP

Everything is now set up to write PHPT tests or just explore and hack the PHP source code. If you want to configure and compile PHP, use the following commands:

# ./configure && make && make test

If you also want to enable code coverage results, use the following command:

# ./configure --enable-gcov && make && make lcov

Optional, you can also pass the TESTS variabele to the make lcov command to only run a specific test-case (of course, this speeds up compiling the PHP source):

# make lcov TESTS=ext/foo/tests/bar.phpt

Next up, have a look at the lcov_html/ directory, where you can find the HTML code coverage results. Open your browser to view the code coverage, and write some PHPT tests to increase the overall coverage!

8. Some links to get you started on testing

PHP-QA: http://qa.php.net
PHP TestFest: http://wiki.php.net/qa/testfest
PHPT: http://phpt.info/

Update (09/07/2010): I’ve updated the article to reflect the change from CVS to SVN as version control system of the PHP project.