Symfony2 on Windows Azure Websites

With the Windows Azure Websites platform-as-a-service (PaaS), which was released as a preview to a general audiance in June last year, PHP applications can be deployed to Azure with as much as a Git push to a remote repository. At the same time Microsoft released a new and much improved PHP SDK, which integrates with all the Azure webservices.

I blogged about deployment with Composer on Azure Websites last November and since then I have been working with Microsoft to improve Symfony2 support on Windows Azure by developing and releasing the AzureDistributionBundle. This is a new version of the bundle and contains the following improvements:

• Full support for the PHP SDK through the DependencyInjection container
• Installation of project dependendencies using Composer during deployment. I will blog about the Composer support in much more detail in the next days, because this information is not only valuable for Symfony2 projects.
• a full demo application (derived from symfony-standard) to show PHP, Symfony & Azure integration
• documentation of the Windows Azure Websites deployment process and possiblities for PHP configuration
• a Stream Wrapper for Azure Blob Storage, currently being reviewed for inclusion into the Azure PHP SDK itself.

You can install both the Bundle or the Demo application via Composer. For the Demo application call:

$ php composer.phar create-project beberlei/symfony-azure-edition

The README includes more information about how to deploy the demo to your Azure websites account.

The bundle can be installed using the following composer.json:

{
    "require": {
        "beberlei/azure-distribution-bundle": "*"
    },
    "repositories": [
        {
            "type": "pear",
            "url": "http://pear.php.net"
        }
    ],
}

Registering PEAR is necessary, because the Azure PHP SDK depends on some PEAR components.

Traits are Static Access

In a Twitter discussion yesterday I formulated my negative opinion about traits and Matthew asked me to clarify it:

I used to look forward to traits as a feature in PHP 5.4, but after discussions with Kore I came to the conclusion that traits are nothing else than static access in disguise. They actually lead to the exact same code smells. Familiar with the outcome of too much static use, we should reject traits as just another way of statically coupling your code to other classes.

Let’s step back and take a look at the code smells that Static code produces:

• Tight coupling, no way to exchange at runtime
• Not mockable
• Static code cannot be overwritten through inheritance
• Global state (increases likelihood of unwanted side effects)

This blog post shows that Traits actually have the first three problems themselves and exhibit the same code smells. But they even have some additional problems on their own:

• Not testable in isolation
• Theoretically stateless, but not enforced in PHP
• Very high impact on the code base (Code-Rank)

Take the following code, which tries to implement reusable controller code through traits:

<?php
class MyController
{
    use Redirector;

    protected $container;

    public function __construct($container)
    {
        $this->container = $container;
    }

    public function someAction()
    {
        return $this->redirect("route_xyz", array());
    }
}

trait Redirector
{
    public function redirect($routeName, $parameters)
    {
        return new RedirectResponse(
            $this->generateUrl($routeName, $parameters)
        );
    }

    public function generateUrl($routeName, $parameters)
    {
        return $this->container->get('router')->generateUrl(
            $routeName,
            $parameters
        );
    }
}

Lets spot the problems:

1. Redirector is tightly coupled to MyController. There is no way to change this during runtime, by using a different type of redirector, it has to be exactly the trait used at compile time. This is exactly what static access enforces as well.

2. The trait accesses $this->container without defining it and couples itself against the implementing classes. We can actually refactor this to include an abstract method getContainer() in the trait. But if we have multiple traits now, all having a getContainer() method then we run into method class problems that cannot be solved. We could pass the container as an argument to the method, but that actually defeats the purpose of the abstraction here.

   Traits using state of their “parents” usually create bidirectional coupling between classes, something which should be avoided for good software design.

3. No way to overwrite subset of functionality. If I want to use only one method of a trait and a second one slighty different, then I cannot overwrite this function of Redirector for example in MyController.

4. I cannot mock the traits functionality, therefore I cannot test MyController as a unit only in combination with a trait.

5. I cannot test the trait as a unit, I always have to create a class that uses the trait to be able to write a test for it. This prevents me from testing traits in isolation.

6. Once you start using Redirector in many controllers, its impact on your code base (see Code Rank) increases a lot. Traits are concrete implementations and therefore violate the Dependency Inversion SOLID principle: Changes in the trait require adoptions in all the implementing classes.

   With aggregation you could depend on an abstraction Redirector or turn it into an abstraction in the moment that you need different functionality.

The discovery of this properties of traits me to the conclusion that traits are just static access in disguise.

To see this argument a bit more drastically, you can “rewrite” a PHP 5.4 trait into “pseudo” static code:

<?php
class MyController
{
    public $container;

    public function __construct($container)
    {
        $this->container = $container;
    }

    public function someAction()
    {
        return Redirector::redirect("route_xyz", array());
    }
}

class Redirector
{
    public function redirect($routeName, $parameters)
    {
        return new RedirectResponse(
            self::generateUrl($routeName, $parameters)
        );
    }

    public function generateUrl($routeName, $parameters)
    {
        return $this->container->get('router')->generateUrl(
            $routeName,
            $parameters
        );
    }
}

Calling dynamic methods statically actually works right now (and access to $this of the parent class will luckily be removed in PHP 5.5). Let’s reformulate it into something that is actually using static methods and will work on 5.5, requires changes to the visibility of properties though:

<?php
class MyController
{
    public $container;

    public function __construct($container)
    {
        $this->container = $container;
    }

    public function someAction()
    {
        return Redirector::redirect($this, "route_xyz", array());
    }
}

class Redirector
{
    public static function redirect($thiz, $routeName, $parameters)
    {
        return new RedirectResponse(
            self::generateUrl($thiz, $routeName, $parameters)
        );
    }

    public static function generateUrl($thiz, $routeName, $parameters)
    {
        return $thiz->container->get('router')->generateUrl(
            $routeName,
            $parameters
        );
    }
}

Can you see the familiarity? If Traits can be rewritten as calls to static methods, how can they be any better than static methods? They exhibit the exact same problems and produce the same code smells.

Conclusion: Traits should be avoided at all costs, just like static methods.

Rule of Thumb: If you want to use a trait, try to think how to solve the problem with aggregation.

If you want to read more about problems with traits, Anthony wrote about them quite a while ago.

Lessons learned: A failed side project

Side projects have always been an important part of how I learn new skills and technologies. This usually ends with me dumping some prototype on Github, sometimes even with an open source project that I commit to maintain over a long time.

Two years ago, for the first time, I pursued a project idea which should not be open-source but a commercial SAAS product. It grew out of the Doctrine teams desire to synchronize Github Pull Requests with our Jira instance: A workflow engine for HTTP services. If this, then that (IFTTT) already existed at that point, but was far too limited for my use-case. I wanted something to allow:

• Accept a Github Pull Request Event
• Open a Jira Ticket
• Comment back on the Pull Request with a link to the Pull Request
• Optionally mention that the Pull Request should be made to master, instead of $BRANCH

I started developing this in my free time based on PHP, Symfony2, CouchDB, PostgreSQL and Backbone.js and got a prototype working early. However instead of reaching the state where I could release the project to others I hit some hurdles:

1. The project had a UI where you could add/remove and reconnect tasks through a graphical workflow editor. The javascript became very messy fast, because I didn’t understand Backbone fully and also didn’t know patterns for decoupling and testing Javascript code.
2. I had to restart with the core domain service side code, because I based it on the Zeta Components workflow library and it was too unflexible for my special requirements, messing the code up.
3. I wanted to implement way too many features and had a huge backlog of issues to implement before “beta”.

Last year in July, when I finally had something remotely usable Zapier hit the market with a beautiful product and support for gazillions of services. At that point my service just had support for Github, Twitter and Jira and for generic HTTP POST requests and a UI that could not be operated by anyone else but me. I was quite demotivated that day I found out about Zapier.

Nevertheless I continued to work on the project and tried to make it even more powerful than Zapier, by introducing more features that neither IFTTT nor Zapier had. Adding this complexity ended up being the nail into the coffin of the project.

Each week I worked less and less on the project, because I couldn’t find the motivation. When Zapier got funded in October I was both sad and happy: Apparently my idea was a good one, however somebody else executed this idea much better than myself. I stopped working on the project that week.

Today I took the day to migrate the Doctrine Pull Request workflow to a simple hardcoded application and disabled my projects website entirely.

I want to use this moment to share my personal lessons learned:

• Choosing a scope that allows you to finish a working prototype within 1-2 months is an important key to success.

  The longer it takes to get something working and useable for others, the less motivation you have. I know from my open-source experience that people using your project is a huge motivation boost.

  The side projects I started since last October are much smaller in scope.

• You can actually get burned out by a side project: by designing its scope way too big.

• You cannot compete feature-wise with startups that put their full attention into a project from morning to night. Either make something small working better than commercial products or quit your job and put your full time into this.

• Side projects are either about learning new technologies or about trying to build something commercially successful. Don’t try to combine this or you might get frustrated by choosing the wrong technology for the job.

• Never ever use an existing open-source library as the core of your complicated business domain. If your domain is something remotely interesting you will fail to achieve your goals with the restrictions of the library.

• Starting a big project alone is not a good idea. I found out that discussing ideas with people is very valuable and at the point where I started sharing my idea with others I was already too far into the project to be able to take most of the advice into account.

• Keeping a project idea secret is completely useless. Others will come up with the same idea regardless. People have ideas all the day, however nobody ever has time to implement them. When they do, execution is even more important than the idea itself.

What are your lessons learned from failed side projects? I would be happy to hear other stories.

Code & Coffee Bonn

On last years SoCraTes Conference I had the pleasure to meet Sandro Mancuso from the London Software Craftsmanship Community (LSCC). Following his Twitter account I stumbled upon a meetup concept called “Code & Coffee” which the LSCC organizes very regularly. At a “Code & Coffee” developers would meet up early in the morning before work to discuss and hack at some coffee place.

I liked the idea very much and started such an event in Bonn. Our first test-event was a success, so that we now open up the concept to anyone who wants to join.

There is a Google+ Community Code & Coffee Bonn where you can join and get regular invites to the events.

Our next event is on March 26th. For more details see the Google+ page.

Looking forward to seeing you there!

On Taming Repository Classes in Doctrine

Over at the easybib/dev Blog Anne posted an entry about their usage of Doctrine Repositories with a growing amount of query responsibilities. I want to respond to this blog post with two alternative approaches, because I have seen the easybib approach multiple times in different projects by different teams and think it can be approved upon a lot.

The problems with the approach outlined are:

• The Repository API does not hide implementation details of the ORM, the QueryBuilder API is returned to the client code. This might seen like nitpicking, however it leads to bloated client code doing the query builder work over and over again. For example the ->getQuery()->getSingleResult(AbstractQuery::HYDRATE_ARRAY) call.

• Different parts of the QueryBuilder filtering cannot be composed together, because of the way the API is created. Assume we have the filterGroupsForApi() call, there is no way to combine it with another call filterGroupsForPermissions(). Instead reusing this code will lead to a third method filterGroupsForApiAndPermissions().

  This can lead to combinatorial explosion of methods that the developer using the repository needs to know. And wading through a list of 100 methods to find the right one is never fun, most importantly when the naming of methods is imprecise.

Generally introducing a new object such as a repository should pass the “Composite is simpler than the sum of its parts” rule. However the approach also clearly demonstrates a bad abstraction. In OOP the primary goal is avoiding changes to affect the whole system.

<?php
class UserCriteria
{
    public $groupId;
    public $hydrateMode = Query::HYDRATE_OBJECT;
}

<?php
interface UserRepository
{
    /**
     * @param UserCriteria $criteria
     * @return array<User>|array<array>
     ***/
    public function match(UserCriteria $criteria);
}

<?php
/**
 * @param UserCriteria $criteria
 * @return array<User>
 ***/
public function match(UserCriteria $criteria)
{
    $qb = $this->createQueryBuilder('u');

    if ($criteria->groupId !== null) {
        $this->matchGroup($qb, $criteria);
    }

    return $qb->getQuery()->getResult($criteria->hydrateMode);
}

private function matchGroup($qb, $criteria)
{
    $qb->where('u.group = :group')->setParameter('group', $criteria->groupId);
}

<?php
$criteria = new UserCriteria();
$criteria->groupId = $groupId;
$criteria->hydrateMode = Query::HYDRATE_ARRAY;

$groups = $app['orm.ems']['api']
    ->getRepository('EasyBib\Api\Entity\User')
    ->match($criteria);

<?php
class UserCriteria
{
    const PERMISSION_READ = 'read';
    const PERMISSION_WRITE = 'write';
    //...
    public $permissions;
}
class UserRepository
{
    public function match(UserCriteria $criteria)
    {
        // ...
        if ($criteria->permissions !== null) {
            $this->matchPermissions($criteria);
        }
        // ...
    }
}

<?php
class UserSpecification
{
    public $groupId;
    public $hydrateMode = Query::HYDRATE_OBJECT;
    public $permissions;

    public function match(QueryBuilder $qb, $dqlAlias)
    {
        $expr = "1=1";

        if ($this->groupId !== null) {
            $expr = $qb->expr()->and($expr, $this->matchGroup($qb));
        }

        if ($this->permissions !== null) {
            $expr = $qb->expr()->and($expr, $this->matchPermissions($qb));
        }

        return $expr;
    }

    public function modifyQuery(Query $query)
    {
        $query->setHydrationMode($this->hydrateMode);
    }

    private function matchGroup($qb)
    {
        $qb->setParameter('group', $this->groupId);

        return $qb->expr()->eq('u.group', ':group');
    }

    private function matchPermissions($qb)
    {
        // ...
    }
}

<?php
class UserRepository
{
    public function match(UserSpecification $specification)
    {
        $qb = $this->createQueryBuilder('u');
        $expr = $specification->match($qb, 'u');

        $query = $qb->where($expr)->getQuery();

        $specification->modifyQuery($query);

        return $query->getResult();
    }
}

<?php
interface Specification
{
    /**
     * @param \Doctrine\ORM\QueryBuilder $qb
     * @param string $dqlAlias
     *
     * @return \Doctrine\ORM\Query\Expr
     ***/
    public function match(QueryBuilder $qb, $dqlAlias);

    /**
     * @param \Doctrine\ORM\Query $query
     ***/
    public function modifyQuery(Query $query);
}

class AsArray implements Specification
{
    private $parent;

    public function __construct(Specification $parent)
    {
        $this->parent = $parent;
    }

    public function modifyQuery(Query $query)
    {
        $query->setHydrationMode(Query::HYDRATE_ARRAY);
    }

    public function match(QueryBuilder $qb, $dqlAlias)
    {
        return $this->parent->match($qb, $dqlAlias);
    }
}

class FilterGroup implements Specification
{
    private $group;

    public function __construct($group)
    {
        $this->group = $group;
    }

    public function match(QueryBuilder $qb, $dqlAlias)
    {
        $qb->setParameter('group', $this->group);

        return $qb->expr()->eq($dqlAlias . '.group', ':group');
    }

    public function modifyQuery(Query $query) { /* empty ***/ }
}

class FilterPermission implements Specification
{
    private $permissions;

    public function __construct($permissions)
    {
        $this->permissions = $permissions;
    }

    public function match(QueryBuilder $qb, $dqlAlias)
    {
        // ...
    }

    public function modifyQuery(Query $query) { /* empty ***/ }
}

<?php
class AndX implements Specification
{
    private $children;

    public function __construct()
    {
        $this->children = func_get_args();
    }

    public function match(QueryBuilder $qb, $dqlAlias)
    {
        return call_user_func_array(
            array($qb->expr(), 'andX'),
            array_map(function ($specification) use ($qb, $dqlAlias) {
                return $specification->match($qb, $dqlAlias);
            }, $this->children
        ));
    }

    public function modifyQuery(Query $query)
    {
        foreach ($this->children as $child) {
            $child->modifyQuery($query);
        }
    }
}

<?php
$specification = new Spec\AsArray(new Spec\AndX(
    new Spec\FilterGroup($groupId),
    new Spec\FilterPermission($permission)
));

$groups = $app['orm.ems']['api']
    ->getRepository('\EasyBib\Api\Entity\Group')
    ->match($specification);

<?php
interface Specification
{
    // ..
    /**
     * @param string $className
     * @return bool
     ***/
    public function supports($className);
}

<?php

abstract class EntitySpecificationRepository
{
    public function match(Specification $specification)
    {
        if ( ! $specification->supports($this->getEntityName())) {
            throw new \InvalidArgumentException("Specification not supported by this repository.");
        }

        $qb = $this->createQueryBuilder('r');
        $expr = $specification->match($qb, 'r');

        $query = $qb->where($expr)->getQuery();

        $specification->modifyQuery($query);

        return $query->getResult();
    }
}

<?php
class PowerUsers implements Specification
{
    private $spec;

    public function __construct()
    {
        $this->spec = new OnlyPage(1, new AndX(
            new UsersWithInteraction(),
            new EnabledUsers(),
        ));
    }

    public function match(QueryBuilder $qb, $dqlAlias)
    {
        return $this->spec->match($qb, $dqlAlias);
    }

    public function modifyQuery(Query $query)
    {
        $this->spec->modifyQuery($query);
    }

    public function supports($className)
    {
        return ($className === 'EasyBib\Api\Entity\User');
    }
}

$top20powerUsers = new Spec\PowerUsers();

Extending Symfony2: ParamConverter

Symfony2 is an extremely extendable framework, everything is extendable or overwritable through the Dependency Injection Container. The problem developers face is knowing about the extension points and when to use them. If you don’t know the extension points, your Symfony application will end up with code duplication, too much inheritance and very little unit-testable code.

This blog post will be the first in a series, describing Symfony2 extension points that help you achieve clean and duplicateless code. In my experience, using Symfony extension points to avoid code duplication helps you avoid writing thousands of lines of code in your controllers.

<?php
class UserController extends Controller
{
    public function showAction($id)
    {
        $dql = "SELECT u, d, a
                  FROM MyBundle\Entity\User u
                  JOIN u.details d
                  JOIN u.addresses a
                  WHERE u.id = ?1";

        $user = $this->get('doctrine.orm.default_entity_manager')
            ->createQuery($dql)
            ->setParameter(1, $id)
            ->getSingleResult();

        if ( ! $user) {
            throw new NotFoundHttpException();
        }

        return array('user' => $user);
    }
}

<?php
class AbstractController extends Controller
{
    protected function findUser($id)
    {
        $dql = "SELECT u, d, a
                  FROM MyBundle\Entity\User u
                  JOIN u.details d
                  JOIN u.addresses a
                  WHERE u.id = ?1";

        $user = $this->get('doctrine.orm.default_entity_manager')
            ->createQuery($dql)
            ->setParameter(1, $id)
            ->getSingleResult();

        if ( ! $user) {
            throw new NotFoundHttpException();
        }

        return $user;
    }
}

<?php
class UserController extends Controller
{
    public function showAction(User $user)
    {
        return array('user' => $user);
    }
}

<?php
namespace MyProject\Request\ParamConverter;

use Sensio\Bundle\FrameworkExtraBundle\Configuration\ConfigurationInterface;
use Sensio\Bundle\FrameworkExtraBundle\Request\ParamConverter\ParamConverterInterface;
use Symfony\Component\HttpFoundation\Request;
use Symfony\Component\HttpKernel\Exception\NotFoundHttpException;
use Doctrine\ORM\EntityManager;

class UserParamConverter implements ParamConverterInterface
{
    private $entityManager;

    public function __construct(EntityManager $entityManager)
    {
        $this->entityManager = $entityManager;
    }

    public function apply(Request $request, ConfigurationInterface $configuration)
    {
        $id = $request->attributes->get('id');

        $dql = "SELECT u, d, a
                  FROM MyBundle\Entity\User u
                  JOIN u.details d
                  JOIN u.addresses a
                  WHERE u.id = ?1";

        $user = $this->get('doctrine.orm.default_entity_manager')
            ->createQuery($dql)
            ->setParameter(1, $id)
            ->getSingleResult();

        if ( ! $user) {
            throw new NotFoundHttpException();
        }

        $param = $configuration->getName();
        $request->attributes($param, $user);

        return true;
    }

    public function supports(ConfigurationInterface $configuration)
    {
        return "MyProject\Entity\User" === $configuration->getClass();
    }
}

<service id="my_project.user_param_converter"
      class="MyProject\Request\ParamConverter\UserParamConverter">
    <argument type="service" id="doctrine.orm.default_entity_manager" />

    <tag name="request.param_converter" converter="user" priority="10" />
</service>

Doctrine and SOLID

I often get asked how you can use Doctrine2 and implement the SOLID principles at the same time. It bugs me having to reply: It is not really possible. Because Doctrine2 does not support value-objects (or embedded-objects), it is very hard to pull the Single Responsibility Principle off.

These problems are related to the inability to share behavioral code through aggregation and the complexity of state transformations. Combining both, your average entity with 5-15 fields can end up with hundreds or thousands lines of code. The solutions to both problems boil down to minimizing duplication and maximizing clarity.

<?php
use Doctrine\ORM\Mapping as ORM;

/**
 * @ORM\Entity
 * @ORM\HasLifecycleCallbacks
 **/
class Person
{
    /**
     * @ORM\Column(type="datetime")
     **/
    private $createdAt;
    /**
     * @ORM\Column(type="datetime")
     **/
    private $updatedAt;

    public function __construct()
    {
        $this->createdAt = new \DateTime("now");
        $this->updatedAt = new \DateTime("now");
    }

    /**
     * @ORM\PreUpdate
     **/
    public function updatedAt()
    {
        $this->updatedAt = new \DateTime("now");
    }
}

<?php
class Timestamped
{
    private $createdAt;
    private $updatedAt;

    public function __construct()
    {
        $this->createdAt = new \DateTime("now");
        $this->updatedAt = new \DateTime("now");
    }

    public function updatedAt()
    {
        $this->updatedAt = new \DateTime("now");
    }
}

class Person
{
    private $timestamped;

    public function __construct()
    {
        $this->timestamped = new Timestamped();
    }
}

<?php
namespace MyProject\Entity {

    class Order
    {
        public function calculateShippingCosts()
        {
            $calculator = new ShippingCostCalculator();
            $this->shippingCosts = $calculator->calculate($this);
        }
    }
}

namespace MyProject\Entity\Order {

    class ShippingCostCalculator
    {
        public function calculate(Order $order)
        {
            return 0;
        }
    }
}

<?php
class Order
{
    public function calculateShippingCosts(ShippingCostCalculator $calculator)
    {
        $this->shippingCosts = $calculator->calculate($this);
    }
}

Composer and Azure Websites Git Deployment

Continuing my series on PHP PaaS Clouds (Fortrabbit), I turn to Microsoft Azure today. After some research I found out Azure supports post deployment hooks to run Composer and allows you to configure environment variables from the Management console.

Microsoft launched Azure Websites in June this year. It is a platform as a service solution where you can deploy your websites via FTP or Git. With Azure Websites you can avoid having to deal with the complex deployment automation that is necessary for Azure Hosted Cloud Services. As long as your website only requires an SQLServer, MySQL or external APIs you can actually achieve quite a lot already.

For a Symfony2, Silex or any other modern project however you want Composer support during the deployment as long as failures with Composer don’t break your site.

It turns out that Azure Websites - to support other platforms that require compiling - actually has an extremely robust deployment system (as far as I understood its internals). The Git repository is separated from the actual code that is served from the webserver and a number of old checkouts is kept to allow rollbacks through the Web interface. Once a Git push was recognized, Azure websites will execute a build step, and then copy all the files over to a directory with the name of the Git SHA hash. This directory is then symlinked to the document root.

If Composer fails during this step, the website will still be served from the currently running version and you don’t have to face unexpected downtime.

To actually run Composer as a post-deployment tool you have to do some manual work. Create a .deployment file in your project root folder:

[config]
command = "D:\Program Files (x86)\PHP\v5.3\php.exe" build_azure.php

If you are using PHP 5.4, then you probably have to change the command version name to “v5.4”, but I haven’t tested this.

Then you need to create the build_azure.php file that is referenced in the deployment command. The actual implementation is up to you, my version is the most simple one:

<?php
if (!file_exists("composer.phar")) {
    $url = 'https://getcomposer.org/composer.phar';
    file_put_contents("composer.phar", file_get_contents($url));
}

$_SERVER['argv'][1] = "update";
$_SERVER['argv'][2] = "--prefer-dist";
$_SERVER['argv'][3] = "-v";
require "composer.phar";

Instead of using this approach, you could ship composer.phar with your repository for example. You can of course execute additional steps in the build_azure.php, for example warmup caches.

Babysteps with Fortrabbit PHP Cloud

I wanted to host a simple application to play around with the Tent Protocol (see my last blogpost), so I checked through some of the PHP Cloud providers again for the fun of it. One requirement was to deploy from Git and update dependencies from Composer. I found a new provider that i haven’t heard of before, Fortrabbit that supports this so I had to check it out. It has some more features that I really dig:

Fortrabbit provides you with a Git url where you can push your repository. It is directly deployed during the push operation. You can trigger composer updates by having a special syntax in the commit message. The push output informs you about all the steps performed during deployment.

Additionally you can configure environment variables in the web-administration interface that are available in the application through $_SERVER. You can easily use them to configure your application, if its hosted in a public repository. Great to share sample applications on Github and host them from there.

You get SSH access to the machines, where you can take a look at the apache and php error log files. You have vim available. Quite cool and very helpful for any kind of debugging necessary. Deployments overwrite every change you make, so its still a save environment.

The composer support allows for post install scripts, which is cool to perform all sorts of cache warmup and other deployment tasks.

You can host one application for free, however they shutdown after 48 hours if you don’t regularly click on a reset button. Its definitely enough to get small prototypes up and running. From there you can upgrade in many small steps from small plans (10€/month) to bigger ones (80€/month).

A tent.io app: Zelten Bookmarks

Over the weekend I built a very simple bookmarking application for tent.io. I wanted to try the tent functionality to create arbitrary entities of data using something as simple as a “Bookmark”.

The application retrieves and stores bookmarks on your own tent-server, nothing is kept on the applications server and as a user you keep full control over your content (bookmarks).

The code for the application is on Github, as well as the TentPHP library I have written to support the protocol. The application is using PHP, the Silex microframework and MySQL.

Using Assertions for validation

We all know PHP is weakly typed and has some weird type-conversion rules. This can often complicate our code checking for invalid or illegal data. Using if/else clauses we can do all sorts of validation, but with standard coding styles (PSR, Zend, PEAR) we end up with at least 3 lines per check, cluttering the whole code-base.

Take this example from the old deprecated Windows Azure SDK, a method for putting a binary file into Azures blob storage:

<?php
public function putBlob($containerName = '', $blobName = '', $localFileName = '', $metadata = array(), $leaseId = null, $additionalHeaders = array())
{
    if ($containerName === '') {
        throw new Microsoft_WindowsAzure_Exception('Container name is not specified.');
    }
    if (!self::isValidContainerName($containerName)) {
        throw new Microsoft_WindowsAzure_Exception('Container name does not adhere to container naming conventions. See http://msdn.microsoft.com/en-us/library/dd135715.aspx for more information.');
    }
    if ($blobName === '') {
        throw new Microsoft_WindowsAzure_Exception('Blob name is not specified.');
    }
    if ($localFileName === '') {
        throw new Microsoft_WindowsAzure_Exception('Local file name is not specified.');
    }
    if (!file_exists($localFileName)) {
        throw new Microsoft_WindowsAzure_Exception('Local file not found.');
    }
    if ($containerName === '$root' && strpos($blobName, '/') !== false) {
        throw new Microsoft_WindowsAzure_Exception('Blobs stored in the root container can not have a name containing a forward slash (/).');
    }
    // rest of the code here
}

The rest of this components public API methods look about the same. It is a very complete validation of the input data, however its not very readable. Instead you have 6 if branches, which increase the complexity of the method and almost take up half the screen without even getting to the actual code.

The assertion pattern is really helpful here to reduce the complexity of this code and hide the if/throw conditions into little helper methods. Have a look at a refactored method using my Assert micro-library.

<?php
public function putBlob($containerName = '', $blobName = '', $localFileName = '', $metadata = array(), $leaseId = null, $additionalHeaders = array())
{
    Assertion::notEmpty($containerName, 'Container name is not specified');
    self::assertValidContainerName($containerName);
    Assertion::notEmpty($blobName, 'Blob name is not specified.');
    Assertion::notEmpty($localFileName, 'Local file name is not specified.');
    Assertion::file($localFileName, 'Local file name is not specified.');
    self::assertValidRootContainerBlobName($containerName, $blobName);

    // rest of the code
}

This is much more readable, using two custom assertions and some general assertions.

There is one risk here though, you introduce additional coupling to another library/namespace. Especially when you start using this pattern on a large scale, you have to make sure this assertions are VERY stable and unit-tested.

That is why you should extend from Assertion and use your own class when using the Assert library. This returns some control in your hands and even allows you to overwrite the exception class:

<?php
namespace MyProject\Util;

use Assert\Assertion as BaseAssertion;

class Assertion extends BaseAssertion
{
    static protected $exceptionClass = 'MyProject\Util\AssertionFailedException';
}

Symfony, Monolog and different log types

Symfony uses the excellent Monolog library for logging everything related to the symfony application stack. But sometimes you need a different logger that responds differently to the log error levels. Ideally you don’t want to reuse the application log mechanism to avoid messing with the fingers crossed listener of Monolog, that keeps the application log files small in case no errors occur.

This can be easily done with Symfonys MonologBundle with a feature called channel. Every log handler, such as file, syslog, mail or fingers crossed is attached to the default logger called “app”. This is referenced by the dependency injection service called monolog.logger.

If you want to create a different logger service that is responsible for a different type of your application stack, just define it using the channels option:

monolog:
    handlers:
        myfile:
            type: stream
            path: %kernel.logs_dir%/myfile.log
            channels: mychannel

To actually write to mychannel you have to tag all the services that use mychannel with the following tag:

<service id="my_service" class="MyService">
    <tag name="monolog.logger" channel="mychannel" />
    <argument type="service" id="logger" />
</service>

With this approach you are now as flexible as with the default channel app regarding to logging in development, testing and production environments for example. Once there is a channel created through the DIC tag, you can actually fetch it from the DIC with monolog.logger.mychannel in our example. I don’t like this approach too much, because it misuses tags for modifying arguments, where tags are actually for the services themselves. However it is usable and works.

This feature is included starting with Symfony 2.1. It is documented in a cookbook entry, however some insights of this blog post are still inside a pending Pull Request.

FrOSCon 2012

This years Free and Open Source Conference (FrOSCon) took place this weekend in Bonn, St. Augustin. Because I live in Bonn, visiting the conference is a given for very many years now. This year I also did a talk on “Architecture Design Patterns” with Gordon so there was an incentive to show up besides meeting friends, all the Club Mate and good open source coffee.

The most interesting topic in this years conference for me has been the Graylog2 talk by Lennart Koopmann. Graylog is a logging server that uses Elastic Search for persistence and searching. It is sponsored by Xing, a pretty big german based Linkedin clone.

The talk The State of PHPUnit by Volker was interesting to see all the new stuff in PHPUnit 3.7 and that it fixes a bunch of annoyances that I stumbled on myself. I actually realized that they are so deeply rooted in my brain already, that they constitute features for me, not something that could actually be optimized. Well done to Volker and Sebastian to the new version.

Igors talk about building an HTTP server in PHP was interesting to see the stream/socket server APIs of PHP at work. Ever since I took part in the SoCraTes 2012 Conference Maexchen UDP Bot challenge I think network programming is an interesting topic for learning about languages and programming itself.

Volker did a second talk on sunday about Clean Code that was both entertaining and interesting. He was reminding the audience that shipping the code is what actually creates the value in our profession. Hence clean code is not about the beauty of code itself, but its ability to be shippable, changeable and adaptable to the business.

Again FrOSCon 2012 was awesome. Thank you very much especially to all the volunteers that organized this event. It really good event to bring together people from many different open-source communities and I already look forward to the next FrOSCon 2013.

Decoupling applications with Domains Events

In the previous posts I have described 3 architectural patterns for business applications. Applying them to your software helps you decouple the business logic from the application/framework/UI. However given sufficient complexity, you will want to decouple different parts of your business model from each other as well.

As an example, lets think of a batch process in your application that updates orders from a CRM or logistics system:

• You receive an XML with full users and order representations
• You need to update certain user fields
• You need to update certain order fields
• Some orders require creating accounts in different remote systems
• If the user has not confirmed his email yet he should receive an opt-in mail instead.
• If the user confirms his opt-in mail, all outstanding remote accounts are created.

You can build all the steps into a single batch processing service. This will be a particularly huge service and in violation of the Single Responsibility principle.

the part with updating of users and fields has nothing to with the sending of mails and creation of remote accounts. We want to decouple them from each other.

The first obvious choice is to decouple them into distinct services:

• Import Service
• Authentication Service
• Account Generation Service

All these services have dependencies on infrastructure objects, database, mailer and so on. We could inject the authentication and account generation services into the Import Service, but with rising complexity this will lead to a lasagna of services and the execution path will dig deep into this and this is not nearly as tasty as eating real lasagna.

ImportService
|_AuthenticationService
| |_Mailer
| |_Database
|_AccountGenerationService
| |_Database
| \_RemoteFacade
\_Database

This becomes more complicated when transaction semantics need to be taken care of. For example dependencies between mailer and database services. Also you have to take into account that the code in entities and value objects participating in this use-case.

What we want instead is a sequential execution of those nested services, but only if the parent service executed successfully:

ImportService
\_Database

AuthenticationService
|_Mailer
\_Database

AccountGenerationService
|_Database
\_RemoteFacade

We could use an event dispatcher in all of the services to notify each other, but the DomainEvent pattern does this more in a much cleaner way:

Every entity is an event provider and can emit events. Whenever a transaction is committed and an hence an operation is completed successfully, we take all the events emitted from all entities (looking at the identity map for example) and trigger observing event handlers. However if the operation fails, we do not trigger these event handlers.

<?php

class Order implements EventProviderInterface
{
    use EventProvider;

    public function importData(array $data)
    {
        // 1. do something with $data

        // 2. raise event
        $this->raise(new OrderImportCompleted(array(
            "id" => $this->id,
            "data" => $data
        )));
    }
}

interface EventProviderInterface
{
    public function dequeueEmittedEvents();
}

The Event provider trait aggregates all the events, and offers and API for external services to pull them from the entity:

<?php
trait EventProvider
{
    private $emittedEvents = array();

    protected function raise(DomainEvent Event)
    {
        $event->getMessageHeader()->setEntity($this);
        $this->emittedEvents[] = $event;
    }

    public function dequeueEmittedEvents()
    {
        $events = $this->emittedEvents;
        $this->emittedEvents = array();
        return $events;
    }
}

Our infrastructure must then trigger event handlers, based on the event names. It will use dequeueEmittedEvents and making sure the events are not emitted multiple times.

For reasons described below, we want the following command/event chain to happen in our system:

• Command executes
• Entities emit events
• Command transaction succeeds
• Events trigger event handlers
• Event handlers execute more commands
• Restart from 1.

With this approach we can decouple all services from each other and avoid deep nesting in each other. Yet we still have transactional dependencies, by dropping all events when the parent command fails. Transactions over multiple commands will not have ACID properties though, instead you will have to look into BASE transactions that are important in systems with eventual consistency. This is one downside that you need to take into account.

The Domain Event pattern is a prerequisite for full blown CQRS. My LiteCQRS library includes a simple implementation of DomainEvent and EventProvider classes and integration into Symfony and Doctrine ORM. Generally this pattern is very easy to implement though, so that you can just have a look at the implementation and take the best parts for your own.

Building an Object Model: No setters allowed

If you are using an object relational mapper or any other database abstraction technology that converts rows to objects, then you will probably use getter/setter methods or properties (C#) to encapsulate object properties.

Take a look at the default User entity from the Symfony2 FOSUserBundle plugin for example: A colossus of getter/setter methods with nearly no real business logic. This is how most of our persistence related objects look like in PHP. A Rails ActiveRecord such as Redmines “Issue” avoids the explicit getter/setters, however generates accessors magically for you. Nevertheless you have to add considerable amount of code to configure all the properties. And code using these active records becomes ambiguous as well.

Why do we use getters/setters so much?

• Tools generate objects from a database, adding getters and setters automatically.
• Frameworks make them automatically available for database records/rows.
• IDEs can magically create getter/setter pairs for fields.
• We want the flexibility to change every field whenever we want.
• It became natural in OOP to have write access to every field.

However Getters/setters violate the open/closed principle, prevent information hiding and should be considered evil (Long version). Using getters and setters allows to decouples the business logic for setting values from the actual storage. Something that object-orientation was suppose to avoid.

<?php
class Post
{
    public function compose($headline, $text, array $tags)
    {
        // check invariants, business logic, filters

        $this->headline = $headline;
        $this->text     = $text;
        $this->tags     = $tags;
    }

    public function publish(\DateTime $onDate = null)
    {
        // check invariants, business logic, filters

        $this->state       = "published";
        $this->publishDate = $onDate ?: new \DateTime("now");
    }
}

<?php
class Post
{
    public function render(PostView $view)
    {
        $view->id          = $this->id;
        $view->publishDate = $this->publishDate;
        $view->headline    = $this->headline;
        $view->text        = $this->text;
        $view->author      = $this->author->render(new AuthorView);

        return $view;
    }
}

class PostView
{
    public $id;
    //... more public properties
}

<?php
class EditPostCommand
{
    public $id;
    public $headline;
    public $text;
    public $tags = array();
}

<?php
class PostController
{
    public function editAction(Request $request)
    {
        $post = $this->findPostViewModel($request->get('id'));

        // This could need some more automation/generic code
        $editPostCommand           = new EditPostCommand();
        $editPostCommand->id       = $request->get('id');
        $editPostCommand->headline = $post->headline;
        $editPostCommand->text     = $post->text;
        $editPostCommand->tags     = $post->tags;

        // here be the form framework handling...
        $form = $this->createForm(new EditPostType(), $editPostCommand);
        $form->bind($request);

        if (!$form->isValid()) {
            // invalid, show errors
        }

        // here we invoke the model, finally, through the service layer
        $this->postService->edit($editPostCommand);
    }
}

class PostService
{
    public function edit(EditPostCommand $command)
    {
        $post = $this->postRepository->find($command->id);
        $post->compose($command->headline, $command->text, $command->tags);
    }
}

OOP Business Applications: Command-Query-Responsibility-Segregation (CQRS)

Other posts in this series about OOP Business Applications:

• Trying to escape the mess
• Entity, Boundary, Interactor
• Data, Context, Interaction

The last pattern for designing your business logic is called “Command-Query-Responsibility-Segregation” or short CQRS. It has its roots in the “Command Query Separation” principle that was put forward by Bertrand Meyer.

Wikipedia has a very good summary about what this principle is about:

Command-Query-Separation states that every method should either be a
command that performs an action, or a query that returns data to the
caller, but not both. In other words, asking a question should not change
the answer. More formally, methods should return a value only if they are
referentially transparent and hence possess no side effects.

CQRS is an OOP architecture design pattern building on that imperative principle.

At its heart is the separation of Read and Write methods. This can be easily achieved by just having two different service layer classes instead of one. This insight alone is not really helpful, its just moving methods around. Coupled to this change however is the notion, that read and write models don’t necessarily have to be the same:

• Getter/Setters: We don’t need setters in the write model, instead we use explicit methods for each write operation. Entities are also much simpler without bi-directional relationships that read/write models often need. Read-only models (ViewModel) don’t need getters/setters. They could just use public properties or even be arrays.
• ORM Performance, Lazy Loading and Query Restrictions: In a write scenario we mostly need queries by primary key, changing only some objects. In read scenarios we need complex queries, joins and aggregations. A separation prevents both from affecting each other negatively.
• Mapping Data-Transfer objects becomes simple. Instead of sending the whole model back and forth, for the write scenario you define use-case specific tasks and updates. This is much simpler than generic mapping of Data Transfer objects to complex object graphs.
• Transforming SQL directly to a view model in a highly optimized way. Simple concept, but it has huge implications: A pure read-only model is simple to maintain and refactor for any performance requirements that come up.

Furthermore with CQRS our write service methods are not allowed to return data anymore (With the exceptions to the rule of course). This may increase the complexity at first, but it has benefits for scalability and decoupling. Following this principle allows us to turn every command from synchronous to asynchronous processing later without much problems. Services are allowed to throw exceptions and refuse the execution. Proper validation of input data should happen before executing write operations though.

One drawback of CQRS: A naive implementation doubles the amount of classes and services to write. But there are simple shortcuts to avoid having to write a full blown read-model if you don’t apply CQRS religiously:

• Don’t use different data-stores for both models.
• Mapping SQL to PHP arrays and stdClass objects through a simple gateway is very simple. You can easily generalize this to a simple object that gives efficient access to all your data.
• Share parts of the write and read models if you don’t have to compromise much.

The important thing to take away from this separation is, that the read part of your application is seldom the part where business value lies in. Moving this code into a different part of your applications allows you to work with the underlying data-source much more directly, without need for much abstractions.

One thing that CQRS puts forward is the concept of Commands, Command Handlers and Command Bus and how they interact. Commands are simple objects that contain all the data necessary to execute an operation. Each command is processed by exactly one command handler. The Command Bus is responsible to route a command to its appropriate command handler.

<?php
interface Handler
{
    public function handle($command);
}
class CommandBus
{
    private $handlers;
    public function register($commandClassName, Handler $handler)
    {
        $this->handlers[$commandClassName] = $handler;
    }

    public function handle($command)
    {
        $this->handlers[get_class($command)]->handle($command);
    }
}

<?php
class TransferMoneyCommand
{
    public $sourceId;
    public $destinationId;
    public $money;
}

class MoneyTransfer implements Handler
{
    private $accountDao; // ctor omitted

    public function handle($command)
    {
        $source      = $this->accountDao->find($command->sourceId);
        $destination = $this->accountDao->find($command->destinationId);
        $money       = new Money($command->amount);

        $source->withdraw($money);
        $destination->deposit($money);
    }
}

<?php
class MoneyController
{
    public function transferAction(Request $request)
    {
        $commandBus = $this->container->get('command_bus');
        $commandBus->handle(new TransferMoneyCommand(
            $request->request->get('sourceId'),
            $request->request->get('destinationId'),
            new Money($request->request->get('amount')
        ));
    }
}

OOP Business Applications: Data, Context, Interaction

Other posts in this series:

• OOP Business Applications: Trying to escape the mess
• OOP Business Applications: Entity, Boundary, Interactor

The next design pattern for technical system design is called “Data, Context, Interaction”. It’s inventors Trygve Reenskaug and James Coplien call it a new paradigm, in the context of PHPs language constraints it can be classified as architectural design pattern. I came across this pattern through Gordon Oheim. He has also invested quite some time going through the book on DCI and trying to understand this pattern with me.

As in the EBI pattern, DCI separates data objects (Data) from behavior implemented in use-cases (Context). Data objects are never directly involved in these use-cases, but are “casted” to their respective roles that they are taking in the use-case. This can be implemented with aggregation or traits in PHP.

One goal of DCI is to get object-oriented programming to a place, where you can look at a use-case and its roles and be very sure that the program is running correctly, because you can guarantee that there are no unexpected side effects. It also aims at keeping the amount of code required to keep in mind for a given use-case as small as possible, allowing developers to reason about the code with more confidence.

<?php
trait TransferMoneySource
{
    public function transferTo($destination, Money $money)
    {
        $destination->transferFrom($destination, $money);
    }
    abstract public function withdraw(Money $money);
}
trait TransferMoneySink
{
    public function transferFrom($source, Money $money)
    {
        $source->withdraw($money);
        $this->deposit($money);
    }
    abstract public function deposit(Money $money);
}

<?php

class BankAccount
{
    use TransferMoneySource, TransferMoneySink;

    private $balance;

    public function __construct()
    {
        $this->balance = new Money(0);
    }

    public function withdraw(Money $money)
    {
        $this->balance = $this->balance->subtract($money);
    }

    public function deposit(Money $money)
    {
        $this->balance = $this->balance->add($money);
    }
}

<?php
class MoneyTransfer
{
    private $source;
    private $destination;

    public function __construct($moneySource, $moneySink)
    {
        $this->source = $moneySource;
        $this->destination = $moneySink;
    }

    public function transferMoney(Money $money)
    {
        $this->source->transferTo($this->destination);
    }
}

<?php
class BankApplicationService
{
    public function transferMoney($sourceId, $destinationId, Money $amount)
    {
        $source      = $this->objectStorage->find($sourceId);
        $destination = $this->objectStorage->find($destinationId);

        $useCase = new MoneyTransfer($source, $destination);

        $conn = $this->conn->beginTransaction();

        try {
            $result = $useCase->transferMoney($amount);
            $conn->commit();

            return $result;
        } catch(\Exception $e) {
            $conn->rollback();
        }
    }
}

OOP Business Applications: Entity, Boundary, Interactor

Other posts in this series:

• OOP Business Applications: Trying to escape the mess

Continuing the series, I will talk about Entity, Boundary and Interactor (EBI) an architectural design pattern. I first heard about it in a keynote video of Uncle Bob on Ruby Midwest called “The lost years of architecture”. This is also described as Hexagonal architecture or “Ports and Adapters” by Alistair Cockburn.

In this pattern any client of the system talks to the model using a model request object and receives data from the model in form of model responses objects. Additionally when your model talks to any kind of persistence or subsystem it should go through an adapter that is replaceable. For a model architecture designed this way, you can replace any part of the UI oder lower layers by writing new adapters and plug them in.

<?php
class BankAccount
{
    private $balance;

    public function __construct()
    {
        $this->balance = new Money(0);
    }

    public function withdraw(Money $money)
    {
        $this->balance = $this->balance->subtract($money);
    }

    public function deposit(Money $money)
    {
        $this->balance = $this->balance->add($money);
    }
}

<?php
class MoneyTransferRequest
{
    public $sourceId;
    public $destinationId;
    public $amount;
}
class MoneyTransferResponse
{
}

class MoneyTransfer
{
    private $accountDao; // ctor omitted

    public function transferMoney(MoneyTransferRequest $transfer)
    {
        $source      = $this->accountDao->find($transfer->sourceId);
        $destination = $this->accountDao->find($transfer->destinationId);
        $money       = new Money($transfer->amount);

        $source->withdraw($money);
        $destination->deposit($money);

        return new MoneyTransferResponse();
    }
}

<?php
interface AccountDaoInterface
{
    public function find($accountId);
}

<?php
class BankApplicationBoundary
{
    private $applicationFactory;

    public function transferMoney(MoneyTransferRequest $request)
    {
        $connection = $this->applicationFactory->createConnection();
        $connection->beginTransaction();

        try {
            $useCase = new MoneyTransfer($factory->createAccountDao());
            $result = $useCase->transferMoney($request);
            $connection->commit();

            return $result;
        } catch(\Exception $e) {
            $connection->rollback();
            throw $e;
        }
    }
}

<?php
$context = $this->getContext();

// 1. direct invocation
$myService = new MyService();
$context->execute(array('service' => $myService, 'method' => 'doSomething', 'arguments' => $args));

// 2. proxy wrapping
$myService = $context->wrap(new MyService());
$myService->doSomething($args);

<?php
class MyService
{
    public function edit(EditRequest $request)
    {
        $entity = $this->dao->find($request->id);
        $this->dataMapper->transform($request, $data);
    }
}

OOP Business Applications: Trying to escape the mess

I mentally divide the world of programming into two large parts:

1. Library and system programming
2. business applications

The first produces code solving technical problems, often in a beautiful and dedicated way. The second regularly produces mess, hopefully still serving customers by optimizing their business. This differentiation is an exaggeration, but from my experience its far easier to run a greenfield business project into the ground than a new library.

There has to be a reason why so many programmers dedicate much free time to open source projects. It might be empirical prove, that all our business projects don’t make us happy at the end of the day and we have to show ourselves that we can do better.

I enjoy writing business applications more than libraries, but they are also much more difficult to get right in both social and technical dimensions. One motivational driver of my open source activities always was to simplify the technical dimension of OOP business applications.

Setting up Development Machines with Puppet

I wrote about Vagrant last week and mentioned to talk about Puppet in the future. Before I will dive into Puppet in combination with Vagrant I want to share my general experiences with Puppet (beginners level, I just played with this today).

Puppet is a configuration management tool. It uses a declarative language to describe how a system should look like. Upon invocation of puppet it attempts to get the system from its current state into the desired state by installing packages, executing programs/scripts and creating files/directories.

My laptop was still on Ubuntu 10.04 this morning, but for Symfony Live I needed to upgrade to 12.04. This was necessary to run a Virtualbox VM on my laptop that I built on my 12.04 desktop. Additionally I really like the Unity desktop and wanted to get away from the Gnome that is in 10.04.

After a backup and fresh installation of Ubuntu I realized how many things I have to install to get this machine productive:

• LAMP stack + a bunch of other databases and services
• Git, Svn, Hg, ..
• Virtualbox + Vagrant
• Vim with my Dot-Files, including Command-T which requires compilation, and a bunch of development tools such as tree, ack-grep.
• PHP Tools (PHPUnit, PHPCS, Xdebug, Xhprof)
• Git with Author Information and global ignore file
• A bunch of .bashrc initializations
• Some open source projects I always have around (Doctrine, ...)
• Setup automatic backup with my cloud storage provider (Strato Hidrive - they support rsync)
• and I probably forgot a bunch of tools just thinking about this today.

I realized not only virtual machines, but also their hosts (my development machines) could be automatically setup with Puppet.

So instead of saving my .bash_history to remember the steps I automated them using Puppet. In Puppet you define Resources of several types, for example Files, Packages or shell commands. You group these resources together in a .pp file and then invoke puppet apply <file>.pp. You can install Puppet via sudo apt-get install puppet on Ubuntu.

The idea is to define Puppet resources in a way that they are performed once and then the system has this resource. This means you can run puppet as often as you want and it will only activate the missing resources.

I separate the installation process into two puppet files, one that has to be run with root ~/.puppet/devmachine-root.pp and another one that has to be run with my own user ~/.puppet/devmachine.pp.

Because every dev-machine setup is unique, I will just show some examples of my puppet file to get the message across:

# /home/benny/.puppet/devmachine.pp

# VIM Dot Files
vcsrepo { 'vim-dotfiles':
    path     => "/home/$id/.vim",
    ensure   => present,
    provider => git,
    source   => 'https://...'
}

file { 'vim-dotfiles-symlink':
    path   => "/home/$id/.vimrc",
    ensure => link,
    target => '.vim/vimrc',
    require => Vcsrepo['vim-dotfiles']
}

exec { 'vim-make-command-t':
    command => 'rake make',
    cwd     => "/home/$id/.vim/bundle/Command-T",
    unless  => "ls -aFlh /home/$id/.vim/bundle/Command-T|grep 'command-t.recipe'",
    require => Vcsrepo['vim-dotfiles']
}

The three keywords vcsrepo, file and exec are called types. They define what functionality should be executed by puppet. Vcsrepo is a custom type that you can grab from Github.

• The first block uses Git to make sure that in my home directory the ~/.vim directory is a checkout of a given remote git repository.
• The second file block makes sure that ~/.vimrc points to ~/.vim/vimrc using a symlink.
• The third block compiles the Command-T VIM plugin. Inside the exec block you can see the unless condition. This is necessary to prevent the command from being executed whenever devmachine.pp is executed with puppet. The require makes sure this resource is only activated after the dotfiles were installed.

Another nice example is the configuration of Git:

# Configure Git
exec { "git-author-name":
    command => 'git config --global user.name "Benjamin Eberlei"',
    unless => "git config --global --get user.name|grep 'Benjamin Eberlei'"
}

exec { "git-author-email":
    command => 'git config --global user.email "kontakt@beberlei.de"',
    unless => "git config --global --get user.email|grep 'kontakt@beberlei.de'"
}

exec { "git-global-ignore":
    command => "git config --global core.excludesfile /home/$id/.gitignore",
    unless  => 'git config -l --global|grep excludesfile'
}

file { "git-global-ignorefile":
    path => "/home/$id/.gitignore",
    ensure => present,
    content => "*.swo\n*.swp\n"
}

Here A bunch of commands is executed unless some option is already set. At last the global .gitingore is filled with the patterns of Vim temporary files.

I also automated all the other steps listed above, but will spare you the details. For the LAMP Stack + PHP I used this following Github repository as an inspiration. It ships a set of Puppet Modules. You can just put them into your ~/.puppet/modules folder and they are then available in any of your puppet files.

I can now reuse this on the different machines: desktop at home and at work and my laptop. Whenever I install a new tool or change some configuration of my system I can directly put this into puppet and keep the setup synchronized on all the machines I work with.

You can find more detailed resources on the PuppetLabs website:

• Tutorial
• Type Reference
• Type Cheat Sheet

Virtual Machines with Vagrant, Veewee and Puppet

I have been playing with Vagrant these last days, a tool to automate Virtualbox VM management very efficiently. Being always late to new technologies investigating Vagrant now felt about right. I am using Virtualbox for virtualization for about a year now but have build all my boxes manually. I have these machines setup:

• Windows 7
• Ubuntu with lots of crazy PHP builds
• Ubuntu with Oracle, PostgreSQL for Doctrine Unit-Tests

The main benefit from this is that I don’t have to clutter my main machine with packages and dependencies that might potentially break.

To benefit from Virtualization on a larger scale I wanted to investigate automation of box-building and was pointed to Vagrant. This is a ruby tool that allows to copy and modify virtualboxes on the fly. You define a basebox and inherit from this box in your projects. Whenever you work on the project you start a copy of this basebox and install more system-dependencies using Chef or Puppet. That allows you to put the VM into a defined state just for this project. No matter how many different and weird dependencies you need, they will always be in this VM that is just created for the project and destroyed at the end of the day.

I am using Ubuntu (currently 12.04) and tried starting with the Vagrant example. It fails, because their example box uses a more current Virtualbox Guest Additions. To be able to use vagrant, you need a basebox with the virtualbox and guest additions versions matching correctly.

To build your own Virtualbox you can use the Vagrant plugin Veewee. Install it from Github to get all the latest VM templates:

$ git clone https://github.com/jedi4ever/veewee.git
$ cd veewee
$ sudo gem install bundler
$ sudo bundle install
$ alias vagrant="bundle exec vagrant"

You need Ruby and Gems installed for this to work. Veewee can be installed using Gems, but this not necessarily gives you the version with the most recent templates that match your own operating system version.

Now lets define our own basebox definition and copy from an existing template that Veewee provides:

$ vagrant basebox define 'beberlei-ubuntu-12.04-i386-php' 'ubuntu-server-12.04-i386'

This creates a copy from the default ubuntu server template into a folder definitions/beberlei-ubuntu-12.04-i386-php. You can start modifying the files in there to adjust the build process of the virtual machine image. For me its important to modify the definitions.rb and add the following, otherwise building the VMs fails:

:virtualbox => { : vm_options => ["pae" => "on"]},

You can then open up the postinstall.sh and install more packages that you need in your personal basebox. For example a LAMP stack. Do this right after the lines that do apt-get install -y:

export DEBIAN_FRONTEND=noninteractive
apt-get -y install php5 php5-cli php5 mysql-server-5.5 libapache2-mod-php5

If you are done you can start building a virtualbox image with:

$ vagrant basebox build

Make sure not to start typing in the console window that open ups. It is automatically controlled from a ruby script and every change to the keysequence breaks the building. This step takes a while. (Enough to write a blog post while watching a boring football game for example).

When this is done, you can verify and export the VM into virtualbox/vagrant.

$ veewee vbox validate 'beberlei-ubuntu-12.04-i386-php'
$ vagrant basebox export 'beberlei-ubuntu-12.04-i386-php'
$ vagrant box add 'beberlei-ubuntu-12.04-i386-php' 'beberlei-ubuntu-12.04-i386-php.box'

Now this box is also available as Vagrant base box. For example in a project that we want to use with vagrant, do:

$ cd myproject/
$ vagrant init 'beberlei-ubuntu-12.04-i386-php'
$ vagrant up
$ vagrant ssh

Although the blog-title suggests it, Puppet hasn’t been used much up to this point. The use of puppet with Vagrant will be part of a next blog post.

Doctrine switches from LGPL to MIT

tl;dr We moved all the Doctrine PHP projects from LGPL to MIT license successfully in roughly 4 weeks. For that we used a small web-based tool to ask the permission of 358 authors that contributed in the last 6 years.

The Doctrine project has been LGPL licensed since it was started in 2006. When I first started contributing in the project late in 2009 we were constantly asked about changing the license to a permissive license such as MIT or New-BSD. My subjective feeling is that there are much more libraries with permissive licenses than 10 years ago. And there are others that back my feeling up with facts. The fear of getting screwed when using a permissive license has probably declined in favor of the benefits. As Lukas puts it:

Maybe with enough experience you start to realize that it happens close to never that a proprietary fork of an open source project ends up outpacing the original project.

We wanted all the benefits of permissive licenses as well. So at the beginning of this year we as Doctrine project first started to investigate how the task of switching the license could be done. After a short email with the FSF it was clear that we had to get the approval of every single committer or remove their code.

VLC successfully attempted a license change last year from GPL to LGPL, so we we’re hopeful to get this done. After discussing with Lukas we came up with the idea of a tool that helps this process, so I wrote it. It features:

• Import of all commits from one or many Github projects
• Aggregation of commits to authors based on e-mail address from Git log
• Status approved/not-approved for every author.
• An e-mail engine to send a request for approval to every author using the awesome Mailgun service, who has not approved yet. We use Mailgun for fun and to have a way to act on bounces easily.
• Admin functionality to mark commits as “trivial or deleted”.
• An audit trail to account for all changes and who has done them.
• Symfony2/Doctrine2 application with debian packaging using FPM.

This project is not yet open-sourced but will be when I have some time to clean up the hardcoded passwords and write a small README on how to adjust the templates.

After importing the commits of all the LGPL licensed Doctrine projects we realized that we have 358 unique committers and about 40-50 without an e-mail address (from the old SVN days). So we started googling for nicknames and eventually collected all e-mail addresses except maybe 6-8 missing ones.

Now after 4 weeks of bi-weekly e-mail reminders, everyone except 16 committers have accepted the license change. No single person refused the change. All commits of the 16 people left we’re luckily not part of the code base anymore when we completely rewrote Doctrine 2 or trivial one-line changes.

I am very happy to announce that Doctrine is now an MIT licensed project. Have fun with it.

Symfony2 Controller Testing without Application

Controller testing using the WebTestCase requires a Symfony Kernel and with that a complete application. However you can just ship your own simple kernel with just the dependencies necessary to test your application. This way you can easily create functional tests for bundles without the bundle requiring an application.

<?php
// Tests/Controller/App/AppKernel.php

use Symfony\Component\HttpKernel\Kernel;
use Symfony\Component\Config\Loader\LoaderInterface;

class AppKernel extends Kernel
{
    public function registerBundles()
    {
        $bundles = array(
            // Dependencies
            new Symfony\Bundle\FrameworkBundle\FrameworkBundle(),
            new Symfony\Bundle\SecurityBundle\SecurityBundle(),
            new Symfony\Bundle\MonologBundle\MonologBundle(),
            new Symfony\Bundle\TwigBundle\TwigBundle(),
            new Sensio\Bundle\FrameworkExtraBundle\SensioFrameworkExtraBundle(),
            new JMS\SerializerBundle\JMSSerializerBundle($this),
            new FOS\RestBundle\FOSRestBundle(),
            // My Bundle to test
            new Beberlei\WorkflowBundle\BeberleiWorkflowBundle(),
        );

        return $bundles;
    }

    public function registerContainerConfiguration(LoaderInterface $loader)
    {
        // We don't need that Environment stuff, just one config
        $loader->load(__DIR__.'/config.yml');
    }
}

# Tests/Controller/App/config.yml
framework:
    secret:          secret
    charset:         UTF-8
    test: ~
    router:          { resource: "%kernel.root_dir%/routing.yml" }
    form:            true
    csrf_protection: true
    validation:      { enable_annotations: true }
    templating:      { engines: ['twig'] }
    session:
        auto_start:     false
        storage_id: session.storage.filesystem

monolog:
    handlers:
        main:
            type:         fingers_crossed
            action_level: error
            handler:      nested
        nested:
            type:  stream
            path:  %kernel.logs_dir%/%kernel.environment%.log
            level: debug

# Tests/Controller/App/routing.yml
BeberleiWorkflowBundle:
    resource: "@BeberleiWorkflowBundle/Controller/"
    type:     annotation
    prefix:   /

<?php
// Tests/bootstrap.php
$loader = @include __DIR__ . '/../vendor/.composer/autoload.php';
if (!$loader) {
    die(<<<'EOT'
You must set up the project dependencies, run the following commands:
wget http://getcomposer.org/composer.phar
php composer.phar install
EOT
    );
}
\Doctrine\Common\Annotations\AnnotationRegistry::registerLoader(array($loader, 'loadClass'));

spl_autoload_register(function($class) {
    if (0 === strpos($class, 'Beberlei\\WorkflowBundle\\')) {
        $path = __DIR__.'/../'.implode('/', array_slice(explode('\\', $class), 2)).'.php';
        if (!stream_resolve_include_path($path)) {
            return false;
        }
        require_once $path;
        return true;
    }
});

<!-- phpunit.xml.dist -->
<phpunit bootstrap="Tests/bootstrap.php">
    <php>
        <server name="KERNEL_DIR" value="Tests/Controller/App" />
    </php>
</phpunit>

Resources for a PHP and Hudson CI Integration

Yesterday I finally had the time to setup my first continuous integration environment. Possible solutions for CI are phpUnderControl, Hudson or Arbit. Although phpUnderControl is the most wide-spread, but from I heard complex to setup/maintain, solution [STRIKEOUT:supposedly a hack] and Arbit just in an early Alpha I decided to give Hudson a shoot. Another reason for this decision, I heard it has a simple plugin architecture and is easy to install and use. Additionally Hudson is easily integrated into Netbeans and Redmine, and I use both tools regularly in development.

My motivation to dive into CI is easily explained. I just never felt it was necessary to add a continuous integration environment to my projects, since I had one or two simple bash scripts that did the job. In general this is rather annoying, because they mostly only run PHPUnit and have to be done using a cronjob or manually, without any real process of notification. Additionally you have no way to navigate the test-results, code-coverage and no history of the last builds. For projects like Doctrine 2 we have the additional requirement to support 4 different database platforms, i.e. 4 different PHPUnit configurations. Currently that is solved by me using a Bash script that iterates over the configuration file names and invokes PHPUnit.

There are already some awesome sources how to get Hudson and PHP working. I’ll list them here:

• Sebastian Bergmann’s Howto in Screenshots
• David Luhman’s series on PHP and Hudson
• Justin Palmers Guide to install PHP and Hudson (With Screenshots)

All those guides are already awesome and I would recommend choosing one of those to install Hudson, I think i can’t add anything new to those. I have used Sebastians Howto, however i also like the third one. David Luhmans guide adds lots of details that are important to get the different parts of a build process to work.

Now what these tutorials all do is that they use a bash command to execute the build process or specify an Ant Build file. However there is also a Phing Build process plugin for Hudson that allows to specify the build.xml targets to execute in the process. From the “Available Plugins” list you can choose the “Phing plugin”.

After installation you have to configure the Phing install. The Phing Plugin Wiki Page shows how to do this. You have to go to “Manage Hudson” => “Configure System” and look for Phing. There you find the dialog to configure your phing installations.

In the context of choosing a build script for your project you can now choose “Phing” instead of Ant:

You can enter the targets to build, for example on my local Hudson instance I only execute “test” for Doctrine 2, since I am not interested in the building and deployment onto the PEAR channel at this development stage.

From inside Netbeans you can then start builds by pointing to the Hudson instance. See this tutorial by one of the Hudson + Netbeans Developers. You can then start all the builds from inside Netbeans and be notified of the success or failure.

Testing Database Locks with PHPUnit and Gearman

For the Beta 2 release of Doctrine 2 we plan to integrate pessimistic Database-level locks across all the supported vendors (MySQL, Oracle, PostgreSql, IBM DB2 so far). This means row-level locking as defined in the ANSI SQL Standard using “SELECT .. FOR UPDATE” will be available optionally in DQL Queries and Finder methods. The Implementation of this extension to SELECT statements is rather trivial, however functional testing of this feature is not.

A general approach would look like this:

1. Run Query 1 and 2 with FOR UPDATE into the background
2. Have both queries lock the row for a specified time x (using sleep)
3. Verify that one of the two processes/threads runs approximately 2*x the lock time.

Since PHP does not support process forking or threads naturally you run into a serious problem. How do you execute two database queries in parallel and verify that indeed one query is locking read access for the second one?

Side note: There are some drawbacks to this testing approach. It could be that one background threads finishes the lock sleep already when the second just starts. The locking would work in these cases, however the lock time would not nearly be 2*x seconds, producing a test-failure. We are talking about a functional test though and I will accept a failure from time to time just to be 99% sure that locking works.

Solving this problem with Gearman provides a pretty nice “real-world” example for the Job-Server that I wanted to share. This blog post contains a stripped down code-example from the Doctrine 2 testsuite. If you are interested, you can see the complete Gearman Locking Tests in on GitHub.

Gearman allows to register worker processes with the job-server and offers clients to execute jobs on those workers in parallel. After installing the Gearman job-server and PHP pecl/gearman extension (Rasmus Lerdorf has a post on installation) we can go on writing our locking tests with Gearman.

The first bit is the worker, a PHP script that tries to acquire a database lock and then sleeps for a second. The return value of this script is the total time required for acquiring the lock and sleeping.

class LockAgentWorker
{
    public function findWithLock($job)
    {
        $fixture = $this->processWorkload($job); // setup doctrine in here

        $s = microtime(true);
        $this->em->beginTransaction();

        $entity = $this->em->find($fixture['entityName'], $fixture['entityId'], $fixture['lockMode']);

        sleep(1);
        $this->em->rollback(); // clean up doctrine

        return (microtime(true) - $s);
    }
}

The glue-code for the worker script contains of the registering of the worker method with the job-server and a simple infinite loop:

$lockAgent = new LockAgentWorker();

$worker = new \GearmanWorker();
$worker->addServer();
$worker->addFunction("findWithLock", array($lockAgent, "findWithLock"));

while($worker->work()) {
    if ($worker->returnCode() != GEARMAN_SUCCESS) {
        break;
    }
}

We need two running workers for this to work, since one worker only processes one task at a time. Just open up two terminals and launch the php scripts. They will wait for their first task to process.

Now we need to write our PHPUnit TestCase, which will contain a GearmanClient to execute two of the “findWithLock” in parallel. Our locking assertion will work like this:

1. Register two tasks for the “findWithLock” method that access the same database row.
2. Register a completed callback using “GearmanClient::setCompleteCallback()” that collects the run-time of the individual workers.
3. Execute this tasks in parallel using “GearmanClient::runTasks()”.
4. Assert that the maximum run-time is around 2 seconds (since each worker sleeps 1 second)

The code for this steps could look like:

class GearmanLockTest extends \Doctrine\Tests\OrmFunctionalTestCase
{
    private $gearman = null;
    private $maxRunTime = 0;
    private $articleId;

    public function testLockIsAcquired()
    {
        // .. write fixture data into the database

        $gearman = new \GearmanClient();
        $gearman->addServer();
        $gearman->setCompleteCallback(array($this, "gearmanTaskCompleted"));

        $workload = array(); // necessary workload data to configure workers
        $gearman->addTask("findWithLock", serialize($workload));
        $gearman->addTask("findWithLock", serialize($workload));

        $gearman->runTasks();

        $this->assertTrue($this->maxRunTime >= 2);
    }

    public function gearmanTaskCompleted($task)
    {
        $this->maxRunTime = max($this->maxRunTime, $task->data());
    }
}

Now if both workers are waiting for processing the task we can run this test and get a green bar for a working lock support.

Immutable DateTime Objects

One serious drawback of PHPs DateTime extension is the mutability of its instances. It can lead to serious bugs if a DateTime instance is passed between different functions and is modified at unexpected places. Additionally this possibly rules out several optimizations for scripts that make very heavy use of dates and could share references to equal timestamps.

Warning: All the code assumes that you work with one timezone only!

The following code is an immutable version of PHP’s DateTime. All setter methods throw an exception and add(),sub() and modify() clone the current instance, apply the operation and return the new instance.

<?php
namespace Whitewashing\DateTime;

class DateTime extends \DateTime
{
    /**
     * To prevent infinite recursions
     *
     * @var bool
     */
    private static $_immutable = true;

    public function add($interval)
    {
        if (self::$_immutable) {
            self::$_immutable = false;
            $newDate = clone $this;
            $newDate->add($interval);
            self::$_immutable = true;
            return $newDate;
        } else {
            return parent::add($interval);
        }
    }

    public function modify($modify)
    {
        if (self::$_immutable) {
            self::$_immutable = false;
            $newDate = clone $this;
            $newDate->modify($modify);
            self::$_immutable = true;
            return $newDate;
        } else {
            return parent::modify($modify);
        }
    }

    public function sub($interval)
    {
        if (self::$_immutable) {
            self::$_immutable = false;
            $newDate = clone $this;
            $newDate->sub($interval);
            self::$_immutable = true;
            return $newDate;
        } else {
            return parent::sub($interval);
        }
    }

    public function setDate($year, $month, $day) {
        throw new ImmutableException();
    }
    public function setISODate($year, $week, $day=null) {
        throw new ImmutableException();
    }
    public function setTime($hour, $minute, $second=null) {
        throw new ImmutableException();
    }
    public function setTimestamp($timestamp) {
        throw new ImmutableException();
    }
    public function setTimezone($timezone) {
        throw new ImmutableException();
    }
}
class ImmutableException extends \Exception
{
    public function __construct()
    {
        parent::__construct("Cannot modify Whitewashing\DateTime\DateTime instance, its immutable!");
    }
}

Its not the prettiest code, but it works.

A next optimization would be a DateFactory that manages DateTime instances by returning already existing instances for specific dates. This is not a perfect solution, since you won’t be able to enforce a single instance when you are using the relative descriptive dates or when calculating with dates using add(), sub() and modify(), however for lots of dates created from a database or other external source it might be quite a powerful optimization depending on your use-case:

namespace Whitewashing\DateTime;

class DateFactory
{
    static private $_dates = array();

    static public function create($hour, $minute, $second, $month, $day, $year)
    {
        $ts = mktime($hour, $minute, $second, $month, $day, $year);
        if (!isset(self::$_dates[$ts])) {
            self::$_dates[$ts] = new DateTime('@'.$ts);
        }
        return self::$_dates[$ts];
    }

    static public function createFromMysqlDate($mysqlDate)
    {
        list($date, $time) = explode(" ", $mysqlDate);
        if($time == null) {
            $hour = $minute = $second = 0;
        } else {
            list($hour, $minute, $second) = explode(":", $time);
        }
        list($year, $month, $day) = explode("-", $mysqlDate);
        return self::create($hour, $minute, $second, $month, $day, $year);
    }
}

This includes some date time calculations and date creation with mktime() and DateTimes unix timestamp capabilities to be able to work. Otherwise the sharing of instances could not be implemented. If you need to create shareable instances from other formats you could just create another creation method for it and convert the format for create() to be used.

Application Lifecycle Management and Deployment with PEAR and PHAR (revisited) UPDATE

Some weeks ago I posted an article on using PEAR and PHAR for application lifecycle management and deployment. Since then I have gotten some feedback through comments, but also discussed the topic with colleagues. I have also optimized the approach quite considerably and even made an open-source project out of parts of it and I want to share all that is new with you. First of all, yes the presented solution was somewhat complex, partly because it is still a proposed idea and definitely up for optimizations. However I am still very convinced of my approach, something I should discuss in more detail.

The only other two languages I have ever programmed something in with more than 50 lines of code are Java and C#. In both languages you can explicitly import different dependencies like libraries and frameworks by adding them to your application, i.e. in java you would add for example Spring as an MVC Web layer, Hibernate as an ORM and several other things to your project and directly bundle them in your executable. This is very easy to configure and maintain in IDEs like Netbeans or Eclipse (C# has the same with allowing you to attach DDLs of libraries to your project). It also makes for a much more straightforward deployment.

In the PHP world this situation was quite different (up to PHP 5.3) for several reasons:

• You could not package a library into a single distributable file.
• The PEAR installer as the only tool for updating and managing dependencies of your application by default installs into a system/global directory. This means dependencies your application uses are located in a completely different location than your application code.
• You can’t manage multiple versions of the same package with PEAR in this system directory, making it very hard to control servers with different applications.
• The global directory with your application dependencies is most often not under version control, which makes deployment of applications with PEAR dependencies somewhat difficult.

There are some solutions to this problems:

• Don’t use PEAR, but put all dependencies in your version control system.
• Don’t use PEAR, and bundle dependencies to your code in the build/deployment process.
• Use PEAR like in the article described, on a per project basis.

The solutions that don’t use PEAR suffer from the disadvantage that you need to keep track of all the library and framework dependencies yourself and upgrade them yourself. This might not be such a huge problem from a first glance, but in my opinion many PHP applications and projects suffer from using either no framework/library or just exactly one. There is no real cherry-picking going on the PHP world, for example I would really like to use Zend Framework for the general application layout, but still use Doctrine for the Modelling and HTML Purifier for the Output Escaping. Certain tasks might then only be solvable with the help of eZ Components, all of which are then to some extend dependencies of my application. With PEARHUB and PEARFARM on the horizon (Read Padraic on this topic) even more PHP code will be distributed using PEAR channels in the near future. My immutable DateTime code for example makes for a great little open source library that could be distributed via PEAR, as well as Yadif - a dependency injection container I am using extensively.

Question: Are you really going to manage all these dependencies correctly manually? Is everything up to date all the time, and upgradeable with ease?

The PEAR driven solution then begins to look very desirable in this light, however it has a considerable disadvantage: The PEAR installer itself works on a system-wide/user-centric basis, making it impossible to manage dependencies of several applications using only one linux user. My little Pearanha to the rescue: I have taken the PEAR installer code (which is distributed with all PHP installations across all systems) and put a new CLI tool on top of it. Its a very simple code-generator that allows to generate a re-configured PEAR installer script which only manages a single application in a given directory. This approach is also used by the symfony plugin mechanism, which internally uses the PEAR installer (did you know?).

Lets revisit my blog application example from my previous PEAR post, first install from Github and make the “pearanha” file executable and put it in your PATH (A PEAR Server Channel will follow any time soon).

Now we need to have an existing application directory somewhere, for example /var/www/blog and then we can put Pearanha on top of it with:

benny@desktop: pearanha generate-project

You then need to specify the project base dir and then the project style (for example Zend Framework or Manual) which prompts your for the directory that should be used for as the vendor/library directory that PEAR will install all the code in. You will also be prompted for a binary/scripts directory which will then hold a new PHP file for you, the file my_phpiranha.

Pro Argument: Switching to Pearanha can be done at any point in your application lifecycle. Just define an additional vendor directory for all the dependencies to go in and generate the applications pear installer and you are good to go.

The generated script is your new application specific PEAR installer and you can begin to install all the required dependencies of your application:

benny@desktop:~$ cd /var/www/blog
benny@desktop:/var/www/blog$ ./vendor/pear/my_pearanha channel-discover pear.zfcampus.org
benny@desktop:/var/www/blog$ ./vendor/pear/my_pearanha install zfcampus/zf-alpha
benny@desktop:/var/www/blog$ ./vendor/pear/my_pearanha channel-discover htmlpurifier.org
benny@desktop:/var/www/blog$ ./vendor/pear/my_pearanha install hp/HTMLPurifier
benny@desktop:/var/www/blog$ ./vendor/pear/my_pearanha channel-discover pear.phpdoctrine.org
benny@desktop:/var/www/blog$ ./vendor/pear/my_pearanha install pear.phpdoctrine.org/DoctrineORM-2.0.0
benny@desktop:/var/www/blog$ ./vendor/pear/my_pearanha channel-discover components.ez.no
benny@desktop:/var/www/blog$ ./vendor/pear/my_pearanha install ezc/ezcGraph

All this stuff is now located in /var/www/blog/vendor. Again you can use PEARs complete upgrade, remove and install functionality for your application, now without the hazzle of having to create a linux user for each project you want to manage this way, which in my opinion is a considerable simplification. The complete application (including its dependencies) can then be put under version control and be readily packaged as a single executable PHAR file by your build process.

As a side node, I did try Pyrus instead of PEAR for the same discussed purpose, however most of the current PEAR channels don’t validate against Pyrus strict standards for the package.xml file. In the future this might change and a Pyrus based application installer will then be integrated into Pearanha.

UPDATE: I renamed PHPiranha to Pearanha as its more appropriate. Also after apinsteins comment on “pear config-create” I rewrote the generate-project parts to use the config-create functionality internally, which allowed me to throw away half of the self-written code. Thanks!

Doctrine 2 Beta 1 released

Today we are happy to announce that the first beta of Doctrine 2 has been released and we fixed 165 issues kindly reported by several early adopters.

You can grab the code from the Github project:

http://github.com/doctrine/doctrine2/tree/2.0.0-BETA1

git clone git://github.com/doctrine/doctrine2.git

Or download it from our website:

http://www.doctrine-project.org/download#2_0

It is our believe that Doctrine 2 brings PHP ORMs to a new level. We are leaving behind the Active Record pattern because we think it hurts testability, project maintainability and is not a suitable abstraction (80/20) for models that exceed the complexity of a blog or otherwise simple web application. Instead we implemented a pure Data Mapper approach with help of the new Reflection functionalities of PHP 5.3, so your Domain Objects neither have to extend a base class nor implement an interface.

We also dropped most of the magical features of Doctrine 1 in favour of a simple and standardized API that is loosely based on the Java Persistence API, a technical standard for Object Relational Mappers. However we try not to blindly follow the “Program PHP like Java” approach and and deviated from JPA where applicable to make the concepts fit better into the PHP environment, such as alternatively hydrating all results into nested array structures for very high read performance.

The cornerstone of Doctrine 2 is the query language DQL. It allows to execute queries on the object level defined by your metadata in a similar fashion to SQL. You can even do Joins, Subselects and Aggregates and Group Clauses in DQL, eliminating the need to circumvent ORMs for more advanced SQL features. Nevertheless it is also possible to write plain SQL and let Doctrine 2 hydrate the results into an object graph.

In the last three month since alpha 4 we have done considerable changes and integrated lots of feedback from our users. The most notable changes are:

• Allowing Constructors of your Domain Objects to have non-optional parameters.
• Allow to define a natural ordering of to Many Collections that is automatically enforced trough an SQL ORDER BY statement when retrieved from the database.
• Shipping the Symfony Console Component to replace our own Console Implementation
• New DQL syntax to load objects partially, omitting potentially expensive fields from retrieval for the current request.
• Changes to how bi-directional have to be defined in the mapping files.
• Several changes to the Events API inside the ORM, to make sure many possible extension scenarios work smoothly.
• Enhancements to our Console Tools
• Surpassed the 1000 unit-tests each running against Postgres, Mysql, Sqlite and Oci8 drivers.
• Moved from SVN to Git: http://github.com/doctrine/doctrine2

We also did several painful backwards incompatible changes that seemed necessary to clean up and optimize the API or allow the ORM to be even faster than before. The beta phase beginning today will not contain any larger BC breaks anymore, opening up this release for a broader testing audience.

For the next iteration several enhancements are planned:

• Support for PDO IBM, IBM DB2, SqlSrv and PDO SqlSrv und MsSql drivers
• Pessimistic Lock Support (FOR UPDATE and SHARED)
• Support for Custom Hydration Modes
• Support for Custom Persister Implementations
• Support for handling very large collections of related objects without needing an in-memory representation of the collection
• Separation of Doctrine\Common, Doctrine\DBAL and Doctrine\ORM into three different projects
• Extend the documentation even further, adding quickstart tutorials, cookbook recipes and enhancing the existing chapters.

We also plan to support several extensions for the 2.0 release such as:

• Migrations
• PHPUnit Database Testing Integration
• NestedSet Support
• Symfony 2 Support
• Zend Framework 2 Support

Please try out the new Beta release If you find the time and leave your feedback in our Issue Tracker, the Mailing-List or come discuss with us on Doctrine 2 on Freenode #doctrine-dev.

New Netbeans PHP CodeSniffer Plugin Version

This morning I took the time to merge several changes by Manuel Piechler into my Github branch of the CodeSniffer Plugin and released a new NBM file for you to download. Here are the changes done by Manuel:

• Ability to specify the path to CodeSniffer Binary
• Automatic Detection of all the installed Coding Standards
• Fixed Automatic Binary Detection on Windows

The only open TODO for this project is the possibility to specify different standards on a per project basis, currently you can only choose a global standard.

Generate Proxy code using a stream wrapper

This blog-post is about an experiment and I am curios what others have to say about it. For Doctrine 2 we need to generate Proxy classes for the mapped objects to implement the Lazy Loading pattern. This is currently done with a console command or during runtime (for development). For Doctrine 2.1 we were thinking about using eval() as a third option for shared hosting or development environments. One annoyance of having to generate proxy code is that you have to regenerate it every time you change the original file. Additionally this makes deployment a little more complicated.

Today I got the idea to use stream wrappers to solve this problem. You can use stream-wrappers in combination with include/require and if you are using APC the generated php code can even be cached. That means you only have to generate the code once and after that everything is served from APCs opcode cache. Additionally by using the return values of stat() for the original file you can automatically regenerate your proxy code in APC when the original file changes.

I haven’t found a good way to pass state into a stream wrapper, that is why I put the data into $GLOBALS before calling include(). The client code looks like this:

<?php
stream_wrapper_register("dc2proxy", "Doctrine\ORM\Proxy\ProxyStreamWrapper");

$proxyDir = "dc2proxy:///proxies";
$GLOBALS['DOCTRINE2_PSW_EM'] = $em;
$GLOBALS['DOCTRINE2_PSW_ENTITYNAME'] = $className;
$GLOBALS['DOCTRINE2_PSW_PROXYCLASS'] = $proxyClass;

require $proxyDir. "/".str_replace("\\", "", $proxyClass) . ".php";

unset($GLOBALS['DOCTRINE2_PSW_EM'], $GLOBALS['DOCTRINE2_PSW_ENTITYNAME'], $GLOBALS['DOCTRINE2_PSW_PROXYCLASS']);

Not the nicest code, but it works. I can generate PHP code and have APC cache it for me until the original code changes. The stream wrapper to make this work looks like this:

<?php
namespace Doctrine\ORM\Proxy;

class ProxyStreamWrapper
{
    private $proxyCode;

    function stream_open($path, $mode, $options, &$opened_path)
    {
        $this->position = 0;
        $this->proxyCode = $GLOBALS['DOCTRINE2_PSW_EM']
                ->getProxyFactory()
                ->getProxyClassCode($GLOBALS['DOCTRINE2_PSW_ENTITYNAME'], $GLOBALS['DOCTRINE2_PSW_PROXYCLASS']);

        return true;
    }

    public function stream_stat()
    {
        $reflClass = new \ReflectionClass($GLOBALS['DOCTRINE2_PSW_ENTITYNAME']);
        $stat = stat($reflClass->getFileName());
        $stat[0] *= -1;
        $stat["ino"] *= -1;
        return $stat;
    }

    public function url_stat()
    {
        $reflClass = new \ReflectionClass($GLOBALS['DOCTRINE2_PSW_ENTITYNAME']);
        $stat = stat($reflClass->getFileName());
        $stat[0] *= -1;
        $stat["ino"] *= -1;
        return $stat;
    }

    function stream_read($count)
    {
        $ret = substr($this->proxyCode, $this->position, $count);
        $this->position += strlen($ret);
        return $ret;
    }

    function stream_write($data)
    {
        $left = substr($this->proxyCode, 0, $this->position);
        $right = substr($this->proxyCode, $this->position + strlen($data));
        $this->proxyCode = $left . $data . $right;
        $this->position += strlen($data);
        return strlen($data);
    }

    function stream_tell()
    {
        return $this->position;
    }

    function stream_eof()
    {
        return $this->position >= strlen($this->proxyCode);
    }

    function stream_seek($offset, $whence)
    {
        switch ($whence) {
            case SEEK_SET:
                if ($offset < strlen($this->proxyCode) && $offset >= 0) {
                     $this->position = $offset;
                     return true;
                } else {
                     return false;
                }
                break;

            case SEEK_CUR:
                if ($offset >= 0) {
                     $this->position += $offset;
                     return true;
                } else {
                     return false;
                }
                break;

            case SEEK_END:
                if (strlen($this->proxyCode) + $offset >= 0) {
                     $this->position = strlen($this->proxyCode) + $offset;
                     return true;
                } else {
                     return false;
                }
                break;

            default:
                return false;
        }
    }
}

What do you think about this approach? Are there any potential problems I am not seeing?

Blog Refactorings with Symfony2, Doctrine 2 and Zeta Components

I have been playing around with Symfony 2 quite a lot lately and have rewritten this blog using Symfony 2, Doctrine 2 and Zeta Components. You can go over to Github and find its project page.

Since I have found Sphinx I am pretty impressed with Restructured Text and have now integrated that as writing language into my backend. Using Zeta Components excellent Document component I transform the written ReST to XHTML. I hooked into the Document rendering and it now supports using the Sphinx directive ”.. code-block:: <language>” and runs the subsequent code through Geshi for highlighting. For example:

<?php
echo "hello world with ReST and Zeta Components!";

Using the jQuery Plugin Tabby and MarkitUp with a ReST extension I can also write more conveniently now. Also to battle spam I have moved the comment system to Disqus.

My next plans for the blog bundle are:

• WebDav support for authoring. All articles will be named “slug.inputformat”, for example “blog-refactorings-with-symfony2-doctrine2-zetacomponents.rst”. For this I plan to write a custom backend for ezcWebdav.
• Making the Bundle simpler to re-use by others.
• More jQuery love to the backend.
• Trigger some events in the blog post cycle and hook a twitter + facebook notification for new posts in there.

Btw: This blog post is really just a bad excuse for me to test the ReST Editor in the backend. I hope you still enjoyed it ;)

Git is great: Helping out with MutateMe

These last days I had fun with Padraic Bradys MutateMe project. It dynamically changes your code (using PECL Runkit extension) and checks weather your test cases fetch the errors occurring from changing the source code. This is a great addition to having a test suite, because it finds all the problematic test cases, that try to do too much and fail to cover the code correctly.

I have also contributed to the MutateMe project already and this was only possible due to Git and GitHub, which makes contributing to any project easy like writing a hello world example in a random programing language. As such I have to agree with Padraic: I love Git, too! He already integrated my and Sebastians changes back into the master and we didn’t have to focus on merging, communication and integration a lot.

My PUMA project is also hosted on GitHub (although nobody has looked into that yet), but I am beginning to write a complete Runner now that calls all three components PHPUnit, PDepend and CodeSniffer at once and directly processes the results. When this is done, I will pack it all up as a PEAR package for anyone to test out.

Unittest and Metrics Aggregator Tool for PHP

Both PHPUnit and PDepend offer export functionality in XML or in a Test-database that is not quite readable for any user from the start. Over the last month I have gradually written a nice webbased tool, that aggregates this data and (todo in the future) relates them to help me with my open source projects.

PHPUnit can be used with a –test-db-dsn command, which saves all information about tests into a Database and PDepend has a strong Package centric source parser for all sorts of project metrics.

What I needed for my Zend Framework related work (and other projects) was a tool that does a run of the complete test-suite for me and saves the information, so that I can see where problems occur (and if they are due to my changes or other peoples). Since PHPUnit will stopped calculating additional metrics, I have also integrated the fabulous PDepend Tool into this aggregator. It shows me, what classes and functions need refactoring due to complexity issues and draws some nice graphs that summarize all sorts of project related information on a per package basis.

You can download or clone the an alpha version at Github. Its written with ezcMvcTools, so you need this to work too. A list of some screenshots is at the Github Wiki page.

Update on Unittest and Metrics Tool

Some days ago i posted about the PHP Unittest and Metrics Aggregator tool that i have written on (and which I have dubbed PUMA). Discussing it with people I came to the conclusion that the approach using ezcMvcTools is quite problematic that it forces to use this application, although the reporting and the application are quite separate. This is not against ezcMvcTools: I love it, its just the wrong type of support.

I began to split up the aggregator into some sort of importing-exporting tool. You can specify library-, test- and output-directory of your application and it will use the tools at hand (PHPUnit, PDepend and CodeSniffer currently) to generate their XML formatted reports. It then parses those XML outputted files and combines their result to give a consistent view on your application.

The Import-To-Output generator uses a three-step approach. An importer emits signals to report observers, which will then hand over their collected data to specific html pages that are then generated to the disc. This is a very flexible approach that allows anyone to extend and re-use the tool to generate project metrics, unittest overview and other interesting details on your application.

Have a look at the tool on Github and play with it. I would really like to hear your thoughts.

Explicit Code requires no comments - Only bad code does

Following the recent discussion on commenting source code by Brandon Savage and Marco Tabini, I wanted to add upon Marcos code example to show that it not even needs any inline comments and can be greatly enhanced in readability with taking 2-5 minutes of time and refactoring the code. The original code sample was:

function __construct(array $pathInfo) {
    $section = $pathInfo[0];
    $file = $pathInfo[1];

    // Assume that the location of this file is inside the main trunk, and that pathInfo has already been filtered.

    $path = dirname(__FILE__) . '/../../' . $section . '/xml/static/' . $file . '.xml';

    if (!file_exists($path)) {
        Controller::singleton()->doError(404);
    }

    parent::__construct('main/xsl/template.xsl', $path);
}

This sample is somehow related to the response and view rendering of an application and its pretty understandable. For me there are some squirks though that would make me have to scroll to other classes to see how they interact. For example the path information falls from heaven. Why is it an array of 2 values and are these sections set the same all the time? And what exactly happens when the front controller does the 404 error? Is there a die() or exit in the doError()

The Agile movement postulates the phase of refactoring for a finished piece of code. Rebuilding the hacked solutions that did it to make them more understandable. I applied this to Marcos code snippet. The refactoring produces about double the code, but its done entirely by using NetBeans nice Rename variable and copy paste on extracting methods, which takes not more than 5 minutes.

class StaticXmlXsltRendererImpl extends AbstractXsltView
{
    function render(array $filteredPathInfo) {
        $templateValuesXmlPath = $this->getTemplateValuesXmlFile($filteredPathInfo);

        if(!$this->templateExists($templateValuesXmlPath)) {
            Controller::singleton()->doError(404);
        } else {
            parent::render('main/xsl/template.xsl', $templateValuesXmlPath);
        }
    }

    private function getTemplateValuesXmlFile($filteredPathInfo) {
        $section = $this->getSection($filteredPathInfo);
        $file = $this->getFile($filteredPathInfo);

        // Assume that the location of this file is inside the main trunk
        return dirname(__FILE__) . '/../../' . $section . '/xml/static/' . $file . '.xml';
    }


    private function getSection($pathInfo) {
        return $pathInfo[0];
    }

    private function getFile($pathInfo) {
        return $pathInfo[1];
    }

    private function templateExists($path) {
        return file_exists($path);
    }
}

In this code snippet the class name, variable names and methods clearly communicate what is done, which wasn’t the case in the previous example. The only change for the clients is the change of using the constructor to using the render() method, which communicates the intend more. Also the render method now clearly communicates that either the error or the parent rendering is executed and leaves no doubt about it.

The getTemplateValuesXmlFile() method still uses the comment to show the assumption about relative paths, but this is only the case because application configuration is made implicit into the code. This has to be extracted to be an explicit configuration constant and the comment can go.

private function getTemplateValuesXmlFile($filteredPathInfo) {
    $section = $this->getSection($filteredPathInfo);
    $file = $this->getFile($filteredPathInfo);

    return APPLICATION_ROOT . '/' . $section . '/xml/static/' . $file . '.xml';
}

In my opinion commenting code is necessary only for non-refactored code that has been hacked into existence and is hard to understand. Either the programmer has to get it done and has no chance to clearly communicate the intend. Or what is even worse the to be changed legacy code is hard to understand but you can’t take the chances to refactor it because its also already in production and has no tests. Now from the second point it is obvious that upon ugly code you put only more and more ugly code to fix the problems. This is what leads to the legacy maintenance problems that pretty much every programmer faces. And then commenting comes into play: You have to add a new feature X and it has to be done fast. You don’t really understand the code or how it works together but you know you can put the new code for the feature into existence but its really unintuitive. So you begin to comment it excessively, because its the only way to clearly show its intend.

There are two mindestting factors that help to write code that is understandable without having to excessively comment it:

• From the beginning, do not write code for the computer but for developers. Changing this attitude really helps to write understandable code like the one above.
• Only leave code better than it was before, never worse.

There are five technical practices that - when followed - allow to write clearly communicating code from the beginning:

1. Giving classes, variables and methods good names. This is a no-brainer but few people seem to follow it anyways.
2. Following the object-oriented Single Responsibility Principle by never giving a class more than one responsibility. Macros example seems to follow this one.
3. Methods should never switch in the level of detail. Micro-work at the datastructure level should never be mixed with macro level delegation to executing large chunks of code. Macros code violates this by mixing the path building micro-level work with the macro-level work of rendering the XSLT template. The path building code can be hidden behind a method to communicate intend more clearly.
4. Exchange if conditions with private methods that explain the condition being checked for. In Marcos example this is not really necessary, because file_exists already is quite a good description to the condition. But in cases of logical combinations of conditions the method extracting is a superior way to explain the conditions intend without having to write a comment.
5. Seperate Query and Command: A method never should do a query which returns the state of an object and a command which executes a set of rules on the state.

These practices sum up to one guideline: Make code explicit. This obviously requires less commenting since a comment of explicit code would be duplication and duplication is bad. What if you have a project that does not follow this guidelines? Then of course comments should be used to explain code, but in the long run this should be refactored to self-explaining code. Additionally every new feature should be programmed explicitly to follow the “leave code better than before” principle.

In my opinion two refactoring tools are missing that would greatly help PHP programmers write nice to read code: Extract method and Replace magic value with constant. Can someone integrate them into NetBeans please?

Update: Fixed a creepy copy-paste code bug, thanks to azeroth for pointing out. Moved methods around a bit to be more reading friendly.

Using a Dependency Injection Container with Zend_Application

Much has been written on Dependency Injection lately, mostly by Padraic Brady (1, 2) and Fabien Potencier (1, 2, 3, 4, 5). My subjective feeling tells me there are now more PHP DI containers out there than CMS or ORMs implemented in PHP, including two written by myself (an overengineered and a useful one). Its an awesome pattern if used on a larger scale and can (re-) wire a complex business application according to a clients needs without having to change much of the domain code. It aims at a complete separation of object instantiation and dependency tracking from the business logic.

Beginning with version 1.8 Zend Framework is able to integrate any of these DI containers into its Zend_Application component easily. The Application component initializes a set of common resources and pushes them into the MVC stack as additional Front Controller parameters. Technically a Zend_Registry instance holds all these resources with their respective resource names as keys. The resources are accessible inside the Front Controller and the Action Controller classes. Assume the resource ‘Db’ is initialized in your application, you can access it with:

$front = Zend_Controller_Front::getInstance();
$container = $front->getParam('bootstrap')->getContainer();
$db = $container->db;

// or:
class FooController extends Zend_Controller_Action {
    public function barAction()
    {
        $container = $this->getInvokeArg('bootstrap')->getContainer();
        $db = $container->db;
    }
}

This is pretty much dependency injection already, but the default registry approach suffers from two serious problem: New instances can only be added to the Container by implementing new Zend_Application resources and these instances cannot be lazy loaded. All resources used by Zend_Application are always loaded on every request. But the application container implementation was developed with Dependency Injection in mind and is not tied to the use of Zend_Registry. Only three magic methods are required by any container that wants to be Zend_Application compliant: __get(), __set() and __isset(). Each instantiated resource is pushed via __set() into the container. If required by another resource, __isset() is used to check whether a resource with the given name exists inside the container and __get() is used to retrieve the instances from the container.

Some month ago I extended Yadif, a lightweight PicoContainer-like DI container for PHP written by Thomas McKelvey, with several features that make it a very powerful DI container in my opinion. It has a detailed documentation and examples on the GitHub Page if you want to check it out. I extended Yadif to be Zend_Application compliant in a way that the application-wide Zend_Application resources can be used as dependencies for objects that are lazy-loaded from the container. Skipping the tech-talk, here is an example. First we have to replace the default Container with Yadif:

$objects = new Zend_Config_Xml(APPLICATION_PATH."/config/objects.xml");
$container = new Yadif_Container($objects);

$application = new Zend_Application(
    APPLICATION_ENV,
    APPLICATION_PATH . '/config/application.xml'
);
// Set Yadif as Container
$application->getBootstrap()->setContainer($container);
$application->bootstrap()
            ->run();

Assume you run Zend_Application with a “Db” and a “Cache” resource. These resources are loaded on every request. The objects configured in Yadif however are not instantiated until they are requested for the first time from the container. We can merge these two worlds and make use of the Application resources inside the Yadif Configuration “objects.xml”, which looks like:

<?xml version="1.0" ?>
<objects>
    <myModel>
        <class>MyModel</class>
        <arguments arg1="db" arg2="cache" />
    </myModel>
</objects>

Instantiating a myModel class inside the Action Controller uses the the Database and Cache resources as constructor arguments:

class FooController extends Zend_Controller_Action {
    public function barAction()
    {
        $container = $this->getInvokeArg('bootstrap')->getContainer();
        $myModel = $container->myModel;
    }
}

Given the possibilites by the Yadif_Container you are now empowered to use Dependency Injection for all your application objects and make use of Zend_Applications resource system. Furthermore any other dependency injection container, simple or complex, can also be integrated easily.

Netbeans PHP CodeSniffer Plugin: Now with Options Dialog

Just two days before the International PHP Conference 09 will start I got an email from alexandrehaguiar that he forked my Netbeans PHP CodeSniffer plugin and added the long awaited Options screen. This is just awesome! This way I don’t have to ask the Netbeans guys at IPC all the questions about how to implement it. I merged the changes back into my branch and released a new version which also fixed another small bug. For me it works with both Netbeans 6.7.1 and 6.8 Beta. I want to thank alexandre very much for this awesome contribution.

What you can do now is chose the Coding Standard you want to use in an Options Dialog under the “Miscellaneous” button. You can choose from a list of pre-defined standards, or even enter your own if you use one. Additionally you can disable that warning are shown, so that with any given coding standard you only see the errors.

|image0|

You can download the new version from the Github Project Downloads page.

Enums in PHP

A colleague of mine complained about the missing support for Enums in PHP and how one often wants to have a variable that is constant and has one of several specific values. We came up with an elegant solution that we want to share, maybe its even helpful to you. If you want to implement Enum behaviour with a simple string or int value you end up with having to validate the values at several different locations in the code rather than being able to strictly enforce the Enum structure by using typehints.

We discussed several approaches to this problem that all seemed a bit ugly in one or another way. For example the SplEnum class is a step in the right direction, however you can only use a type hint for “SplEnum” or add another inheritance layer. Also you have to implement different classes for each enum type, which requires you to implement a factory method for your enum to help with creation of the different types.

We came up with a very simple Enum concept. It uses reflection in the constructor to check if the given value is a valid Enum value by checking it against all the defined constants of the implementing class and throws an Exception if it is not. The __toString() magic method is implemented to allow for simple checks of the enums value. Strict type-checks are not possible with this construct, however in our opinion it is a very elegant solution to enforce a limited set of specific values throughout your code-base.

Here is the code plus a small example:

abstract class MyEnum
{
    final public function __construct($value)
    {
        $c = new ReflectionClass($this);
        if(!in_array($value, $c->getConstants())) {
            throw IllegalArgumentException();
        }
        $this->value = $value;
    }

    final public function __toString()
    {
        return $this->value;
    }
}

class Foo extends MyEnum
{
    const FOO = "foo";
    const BAR = "bar";
}

$a = new Foo(Foo::FOO);
$b = new Foo(Foo::BAR);
$c = new Foo(Foo::BAR);

if($a == Foo::FOO) {
    echo "My value is Foo::FOO\n";
} else {
    echo "I dont match!\n";
}

if($a == $b) {
    echo "a value equals b value!\n";
}
if($b == $c) {
    echo "b value equals c value!\n";
}

Now you could nice things such as:

function doStuff(Foo $foo) {
    switch($foo) {
        case Foo::FOO:
            echo "do more here!\n";
            break;
        case Foo::BAR;
            echo "evil stop!\n";
            break;
    }
}

doStuff($a);

What are your thoughts?

Zend_Form and the Model: Yet another perspective using a Mediator

Matthew Weier O’Phinney of the Zend Framework Devteam wrote a controversial post on integrating Zend_Form and the Model last month. He separated concerns of view and model that communicate via a Form, by calling just thee validation functions on the Form inside the mode. On request you could retrieve the model to the controller and view layers. I already wrote into his comments that I didn’t like the solution because it relies on implicit rules for the developers to use the Form component correctly in all layers. Additionally the building of the form using this approach would be performed inside the model, although strictly speaking this is responsibility of the View Layer. Another negative point is duplication of input filtering code that has to be performed to use certain variables inside the controller or when different forms talk with the same model.

Jani took it up and proposed writing validators for forms and attaching them to the Form as sort of a mediator. I am not a fan of this approach either, because the validator would have to include domain logic but is not really a part of the domain logic anymore but just a validator. Developers might forget using the validator inside the model for all their actions or there would be duplication of code in some places. In a perfect world, only functions of the models public interface should be called for validation.

My personal favorite for Form and Model integration is a mediator object between the two layers. Your model will have to include an additional interface with one function acceptFormRequest($values); which accepts an array of validated Zend Form field values. It then tries to apply the validated data into a record. Additional required validations of the model can take place in this function, which separates the concerns of Form validation and Model data validation. Still the mediator merges those differences together: You can throw an Exception and it will be attached as a custom error message to the Form. The following very short code will show the required interface and the mediator code. This code is very simple and might produce maintenance overhead fast, but I propose some refactoring enhancements later in the discussion.

interface WW_Model_AcceptFormRequest
{
    /**
     * Acceept a form request
     * @param array $values
     * @return WW_Record_Interface
     */
    public function acceptFormRequest($values);
}
class WW_Model_FormMediator
{
    /**
     * Try to push the form request to the model
     *
     * @param Zend_Form $form
     * @param WW_Model_AcceptFormRequest $model
     * @return WW_Record_Interface
     */
    public function pushFormToModel(Zend_Form $form, WW_Model_AcceptFormRequest $model)
    {
        if(!$form->isValid()) {
            throw new Exception("Form not valid!");
        } else {
            $values = $form->getValues();
            try {
                $record = $model->acceptFormRequest($values);
            } catch(Exception $e) {
                // This exception message comes from the model, because validation failed
                $form->addErrorMessage($e->getMessage());
                throw new Exception("Form request not accepted by model!");
            }
        }
        return $record;
    }
}

You can see the mediator has two different stages where errors can occur: When the form is not valid or the model is not valid. Both exits can be catched inside the controller and are the indicator that the form has to be displayed again for further input corrections. When successful the model returns a valid record that applies to the form and model requirements and can be displayed. If this record should be persistent this would have been done inside the acceptFormRequest function already. An example using a very simple Model using the a BankAccount example. We have a form that validates all the incoming request data for a withdrawal of money, though does not validate it against the models internal state. Our BankAccountModel implements the WW_Model_AcceptFormRequest interface and returns a valid BankAccount. If found the given amount is withdrawn.

class BankAccountModel implements WW_Model_AcceptFormRequest {
    public function acceptFormRequest($values)
    {
        $bankAccount = $this->getBankAccountBy($values['bankAccountNumber'], $values['pin']);
        if($values['action'] == "withdraw") {
            $bankAccount->withdraw($values['amount']);
            $this->save($bankAccount);
        } else {
            // unknown action...
        }
    }
    public function getBankAccountBy($key, $password) {
        // Find by Primary Key returning 'BankAccount' instance or exception if not found.
    }
    public function save(BankAccount $ba) {
        // Sql for saving the Bank Account
    }
}

class BankAccount
{
    public function withdraw($amount)
    {
        if( ($this->getBalance()-$amount) < 0 ) {
            throw new Exception("You cannot withdraw more money than your bank account holds!");
        }
        $this->balance -= $amount;
    }
}

Two exceptions might be thrown in this case: The Bank Account number does not exist or the password is wrong. Or you are not allowed to withdraw the given amount of money. If any of those exceptions is thrown the Model does not accept the form data and the form will have to be displayed again for the client showing the new error message that was returned from the model. The controller handling this process would look like this:

class BankAccountController extends Zend_Controller_Action {
    public function performWithdrawlAction() {
        $form = new BankAccountWithdrawlForm(); // extends Zend_Form and builds the form

        if($this->getRequest()->isPost()) {
            $mediator         = new WW_Model_FormMediator();
            $bankAccountModel = new BankAccountModel();
            try {
                $bankAccount = $mediator->pushFormToModel($form, $bankAccountModel);

                $this->view->assign('bankAccount', $bankAccount); // Show new balance in view!
            } catch(Exception $e) {
                $this->view->assign('withdrawlForm', $form);
                $this->_redirect('showWithdrawl');
            }
        } else {
            $this->view->assign('withdrawlForm', $form);
            $this->_redirect('showWithdrawl');
        }
    }
}

You can see the mediator tightly integrates Form and Model without both components knowing too much of each other. Still you can add error messages received from the model into the Form and redisplay it. One negative point of this approach is the fact that you only have one method for accepting form data, which could result in variable checking and redispatching in the case of many different operations that can be performed on the same model. For this case you might want to either:

1. Rewrite the mediator to accept a specific model class (not the interface) and call the required custom method that matches the forms request. (Best approach for separation concerns)
2. Rewrite the mediator to also pass the get_class($form); value to the model for decision making (Faster approach)

There is still some overhead on using the mediator. Since its generic you could build an Action Helper for it and use the direct call mechanism to save some lines of code.

Finally: Zend_Mail charset and/or long lines header encoding bug fixed

There was this lurking bug in Zend_Mail which destroyed every Mail-Header (and corresponding Mail) with non US-ASCII Chars and more than an encoded length of 74 chars. This is quite a huge subset of mails, but it seems a nice solution was not so easy, at least nobody tried to fixed it for quite some time.

Where many hackish solutions we’re offered, Ota Mares aka Littlex spent incredible time to hunt the original problem down and with his help I tag-teamed the bug to death today. Saturo Yoshida of Zend Fame added some further spice regarding an alternative solution with Base64 Encoding instead of Quoted Printable Mime Header encoding.

In the end the solution we chose was, not to re-use the Mime encoding function that is specific to MIME bodies according to RFC2045, but to write a completely new algorithm for Mime Headers, whose rules are specified in RFC2047. This is now done and unit-tests prove its working according to standard.

What is missing now is people trying that fix on as many Mail platforms as possible and giving feedback in the issue if a lengthy subject with non-ASCII chars is displayed correctly.

Howto file a good bug report: Suggestions for framework users

I have fixed quite a number of bugs for the ZF lately, which lead me to this post about how to file a good bug report. There are many annoying bug reports out there, where the reporter of the bug withholds important information to the bugfixer unintended. This advice applies bug reports in general of course.

What are the benefits of a good bug report? The bug generally gets fixed faster, when the developer has more information at hand. Additionally other developers might come to rescue since they can understand the issue faster. These benefits are good for both parties. If you take no time for a good bug report, your issue might risk to end up getting old or closed unfixed.

• Post the whole Exception Stack Trace: If the library throws an Exception into your application that is unexpected and may indicate an bug: Do not post the Message or Exception name only. The exception may be thrown in many different places or due to different reasons. The PHP exception class offers the method getTraceAsString(), which offers many information to the developer what the cause of the exception might be. Please use it!
• Post codefixes in a patch format: When you find a bug in the framework, it is quite possible that you can offer a fix directly. Writing “Replace x in line y with z” does not help very often. The component might be in flux and the line positions change more often than you think. Please create a diff file of this changes that indicate the precise position of the change. This diff also includes 2 lines above and below the patched code for direction of the developer. SVN Diffs are even more useful since they include the revision where you fixed the bug in.
• Post reproducible cases as PHPUnit Test: If you find a bug and can show how to reproduce it: Write a unittest to prove it. It is ZF policy to create a unittest for each bugfix showing that the bug was indeed fixed and previous functionality remains the same, so this unit-test has to be written anyways. Many show-offs rely on massive echo statements or var_dump, which render them almost useless for the developer.
• Attach a unit-test to a submitted patch: This is related to the previous point. If you add a unit test your patch will get more attention. It will prove that you have thought about the patch, its consequences and that you might have checked it does not break backwards compatibility. This is worth a lot.
• Run the test suite with your patch: If you want to provide a patch. Run the testsuite of the Zend Framework. It might break expected behaviour. When you post a patch that will break BC, it will be recognized. Your bug report might be closed, which helps nobody.

When you find a bug you have probably thought about it and how to fix it. This is valuable information. Disregarding one of this points will lead to missing information on part of the developer that he has to “learn” again. This takes time, which may make your bug last longer than it should.

What will be new in ZF 1.8

In February or March 2009 the 1.8 version of the Zend Framework is scheduled to be released. I have contributed some stuff already regarding ZendX_JQuery and Zend_Soap.

Both components have seen numerous bugfixes and I managed to get the JQuery helper down to zero open issues. I have also taken over the Zend_Json_Expr proposal, which will be a huge benefit to everything JSON that can be done with ZF. Foremost it is an integral part for the jQuery component which heavily relies on javascript callbacks.

The Soap Autodiscover and WSDL classes compatibility with Java and .NET has been optimized due to great user feedback, as well as some bugfixes to the newly added WSDL type detection strategies.

Additionally I went on another bug killing spree and fixed around 20-30 old bugs in a wide range of different components.

On Publishing Webservices within MVC Frameworks

Webservices are a very important part in today’s enterprise applications. They tie applications of different age or programming languages together or allow applications of different subcontracters to speak to each other. Because they use HTTP, a stateless network protocol, considerable overhead floods the pipes when you use them, which should be minimized.

Martin Fowler writes in his PEAA book, that if you have the option not to use distributed objects (which are implemented via webservice) you should not distribute them. Considerable effort has to be brought into keeping complex webservices performant.

Still people make mistakes about webservices all the time (me included for example proposing a dispatcher for the ZF that could be used for webservices).

When people report problems with the Zend Soap component they often post a stripped down example that involes their webservice being instantiated within a controller. This is a very bad decision based on different arguments:

• Dispatching overhead: Dispatching, Routing, Pre- and Postfiltering is costly in all frameworks. You give up the performance of having numerous PHP scripts that act as controller on their own. You get centralized filtering, authentication and other benefits. But those benefits generally do not aply to XML, JSON or SOAP requests, because you cannot parse them or access their properties. You give up the performance of a page controller for webservices to gain mostly nothing.
• HTTP Request uselessness: Web frameworks work with HTTP request objects. The request of webservices facilitates HTTP to act as a far more complex request. No framework I know off, allows to work with the webservice requests outside the Webservice handler. What a SOAP or XML-RPC request does in your MVC is only get passed through numerous costly stages that offer no benefit, before it hits the target. Only the parsing of HTTP-Headers might offer additional benefit, but the gain is low, since they are available to PHP scripts at no cost.
• Webservices already seperate concerns: Take the PHP SOAPServer as an example. It is an MVC application on its own, the controlling aspect of the SOAPServer parses the SOAP Request and sends it to the model, a class given by the user, which in turn works and returns the result as an SOAP Response View. You have to decouple model and view for a webservice handler otherwise it would generate invalid responses. Why nest a perfectly separated operation into another one? You gain no more of this additional separation, except performance decrease.

So what are good practices to implement webservices?

• Use a page controller that generates no MVC overhead. In context of the Zend Framework: Add a new php script to your web root and add a new route into your .htaccess file that redirectes the desired location of the webservice to the script that overwrites the standard catch-all incoming requests to the front controller script.
• Use the proxy pattern and the invaluable __call() method to implement wrapper objects for authentication and session management of the webservice. These classes can easily be reused by all webservice page controllers of your site. If you do your homework you can even share parts of these objects inside your Web-MVC application to keep the code DRY. Those proxies keep authentication logic out of your service class.
• Use the remote facade pattern to implement a few, powerful methods that delegate the service request to underlying domain objects. Never ever publish direct access to domain objects with your webservices. As a rule of thumb, talking to a webservice during a logical operation should never involve more than one or two calls. The first call is for data fetching, the second for data saving. Authentication should be handled via HTTP Authentication to save an additional call.

If you follow these simple rules, you should get around the performance issues that generally come with webservices, without loosing flexibility at all.

Seven Things

I have been tagged by Freeaqingme and the blog letter machinery. So read on for seven things you probably may not know about me.

• I will get my degree in some weeks, currently intensively writting on my thesis. My major is economics though. I probably won’t get a job with that sooner or later. I actually don’t want to, I have a job as programmer already.
• One christmas many years ago (Pentium PCs to 200Mhz were the coolest thing around) i wished and wished for a computer for playing Command and Conquer and stuff. What i got was a 3/86 i could only run crap with. So i had to start programming.
• In 11th grade (7 years ago) I took part in the german computer science olympics (exercise still online) using PHP and writing thousands of lines of nested aray and for loop code and print the solution to the browser. No need to say I utterly failed, because I didn’t know a bit about algorithms, pointers and objects at all.
• I have only learned about what good programming practices are (OO, functional, whatever) really means in the last two years, spending almost every free minute of those many i had on projects to test around with patterns and programming styles.
• I went to my first conference last year september, the IPC in Mainz, getting to know lots of great people in the community.
• I don’t have a license at all (Reference to Sara). I am afraid of driving cars, although I drive my bike through the biggest crossroads in town like nuts.
• I am a timeline oriented reader, having made me masterplans to read both all the Discworld and Ian Rankin novels in the correct order to not miss a single reference.

Its sad that everyone I would want to tag was already tagged by someone else, still for the head count:

• Matthew Weierophinney, because he helped me understand Zend Framework alot.
• Ben Scholzen, because he is a fun guy to be around #zftalk.dev.
• Thomas Weidner, because I like his dedication towards his ZF components.
• Kore Nordmann, Bastian Feder and Thomas Weinert, because I had great discussions with him on IPC.
• zomg aka Jani because of his skills to be referenced by PHPDeveloper.org (joke) and the good IRC discussions.
• Petr Pisl because of the awesome NetBeans PHP support.

Using Zend_Soap Server and Autodiscover in a Controller

I am in a dilemma. I have condemned the usage of Zend_Soap_Server (or any other webservice server handler) inside a Model-View-Controller application before. Still I get questions about my old Zend_Soap_Server and Zend_Soap_AutoDiscover example not working in the MVC scenario. The following example provides you with a working Soap server inside a Zend_Controller_Action, although I discourage the use of it and would suggest using a dedicated script outside the dispatching process to gain multitudes of performance, which webservices often require.

require_once "/path/to/HelloWorldService.php";

class MyDiscouragedSoapServerController extends Zend_Controller_Action
{
    public function serverAction()
    {
        $server = new Zend_Soap_Server("http://example.com/pathto/wsdl");
        $server->setClass('HelloWorldService');
        $server->handle();
    }

    public function wsdlAction()
    {
        $wsdl = new Zend_Soap_AutoDiscover();
        $wsdl->setUri('http://example.com/pathto/server');
        $wsdl->setClass('HelloWorldService');
        $wsdl->handle();
    }
}

Now all you have to do is create two routes, one that makes http://example.com/pathto/server point to MyDiscouragedSoapServerController::serverAction and the other route that makes http://example.com/pathto/wsdl point to MyDiscouragedSoapServerController::wsdlAction. The wrong version (Version Mismatch) error comes from sending a request for the WSDL file to the actual Soap Server, which he doesn’t like.

Test your Legacy PHP Application with Function Mocks!

Much talking is going on about Unittesting, Mocks and TDD in the PHP world. For the most this discussions surround object-oriented PHP code, frameworks and applications.

Yet I would assert that the reality for PHP developers (me included) is dealing with PHP 4, PHP 5 migrated, or non-object oriented legacy applications which are near to impossible to bring under test. Still this code works, is in production and needs maintenance and possibly extension for new features.

For example many applications may still use mysql_* functions at length inside their model or have multiple nested function levels (Example: Wordpress). Runkit to the rescue: A PECL extension that can hack your running PHP code, such that method or functions are repointed to execute new implementations. Using this extension you can actually mock out internal PHP functions, which is great to bring legacy code under test.

Consider this following proof of concept, using the Runkit extension build by Padraic Brady (the one from pecl.php.net does not compile on PHP 5.2), which replaces the functionality of mysql_query(). You have to set the following option in your php.ini: runkit.internal_override = On for this to work. By default only user-defined functions may be overwritten by Runkit.

class FunctionMocker
{
    protected $_mockedFuncBehaviourMap = array();

    public function mock($funcName, $return=null)
    {
        $newFuncCode = 'return "'.$return.'";';

        $renamedName = "__".$funcName."_mockOriginalCopy";
        runkit_function_copy($funcName, $renamedName);
        runkit_function_redefine($funcName, '', $newFuncCode);

        $this->_mockedFuncBehaviourMap[$funcName] = $renamedName;
    }

    public function reset()
    {
        foreach($this->_mockedFuncBehaviourMap AS $funcName => $renamedName) {
            runkit_function_remove($funcName);
            runkit_function_copy($renamedName, $funcName);
            runkit_function_remove($renamedName);
        }
        $this->_mockedFuncBehaviourMap = array();
    }
}

$mocker = new FunctionMocker();
$mocker->mock('mysql_query', 'hello world!');

echo mysql_query(); // hello world

$mocker->reset();

mysql_query(); // error

This example, only allows for string return values of the mock and has no support for replacing the arguments of the mocked function. Also chaining of different return values based on input or call number might be interesting. Some kind of Code Generator Tool would have to be implemented to support this functionality. Additionally Assertions and Verifying should be implemented for the function arguments. All in all this would allow to mock functions as you would mock interfaces/classes, which would be a great addition for all those legacy applications that use procedural PHP.

Additionally what the real killer for runkit would be: The possibility to insert PHP callbacks instead of real PHP code into the runkit_function_redefine.

Speaking about Framework Quality on the IPC Spring 09

Just a short notice, my thesis is taking its last breath, I will be speaking on the IPC Spring 09 conference in Berlin end of may this year on Framework Quality and Unittests:

Framework Quality: Unit-Tests as silent witness

Unit-Tests are often overlooked in framework discussions that focus

on features, performance, tools or database abstraction. However they are a silent witness on how poor or well frameworks are designed. In this presentation we will look at the testsuites of the big players in the PHP framework business - Zend Framework, ezComponents, Symfony and CakePHP - and reveal their hidden secrets.

This talk will make heavy use of the PHP Unittest and Metrics Aggregator tool, which i haven written on several times. For both CakePHP and Symfony extensions for their testtools may have to be written.

I am really excited for my first time being speaker on a conference and looking forward to see you at IPC.

PHP CodeSniffer for Netbeans v0.2

I finally found some time to spend some time on the PHP CodeSniffer for Netbeans plugin. Previously the plugin used an unnecessary API which restricted the use to Netbeans 6.7.0 only. This API was removed so that the plugin should now work with all Netbeans Versions >= 6.7.0.

Additionally when working the previous version would scan every PHP script on a file per file basis when the all projects or main projects filters were activated. This rendered made the plugin almost useless. In the current version scans of filters that contain more than one file are blocked. That means you will only see Coding Violations when you enable the “Current File” filter. Using any other filter will just do nothing and won’t put your Netbeans in permanent hibernation mode (aka useless mode).

There is also some preference “phpcs.codingStandard” using the `NbPreferences API <http://bits.netbeans.org/dev/javadoc/org-openide-util/org/openide/util/NbPreferences.html>`_ which allows to configure the coding standard. [STRIKEOUT:However this feature was contributed by Manuel Piechler and I don’t yet understand how I can manipulate it. Maybe someone knows how (*Looking in Mapis general direction*)?] By default the Zend Coding Standard is used.

Update: Coding Standard can be changed by hacking into the config file .netbeans/6.7/config/Preferences.properties setting “phpcs.CodingStandard=”. Additionally I fixed several bugs with the inline highlighting that did not refresh when lines in the file changed.

Trying a Two Step PEAR/PHAR approach to develop and deploy

With PHP 5.3 the PHAR extension is a pretty powerful concept for all your deployment needs, however it does not tell the complete story. Frameworks, Libraries and many different of them are used throughout applications and in recent times people even began to chery-pick the best components from each of the frameworks and package them together. With Pirum being a simple PEAR channel server there is also momentum for projects to distribute their code via PEAR.

However PEAR is mostly used in the server-wide configuration use-case, which is not very useful if you plan to distribute your complete application in one PHAR file. I just recently had the idea for this scenario, so please bear with me and add all the feedback and comments you can come up with. I tested this with the ongoing rewrite of my blog software.

First we’ll add a new user that we develop our application with:

sudo useradd -m -g www-data whitewashing
sudo passwd whitewashing
su - whitewashing

Now there is the possibility that with this user PHP and PEAR is not in your $PATH environment, so you might have to add it. In my case on Ubuntu i also had to switch the console from /bin/sh to /bin/bash for this user. Then we need to setup our application, I am going to use the Zend Framework project style here but with a little twist. We will add a distinction between vendor and project libraries by adding a vendor directory into the main folder.

But first we create a folder for our application, and create a Zend Framework project in the subfolder “trunk”, which will be the focus of our development.

whitewashing@desktop:~$ mkdir whitewashing
whitewashing@desktop:~$ zf create project whitewashing/trunk
Creating project at /home/whitewashing/whitewashing/trunk
whitewashing@desktop:~$ mkdir whitewashing/trunk/vendor

Now we can configure an Apache virtual host to point to our /home/whitewashing/whitewashing/public directory, i call this “whitewashing-dev” add it to my /etc/hosts and can visit the dummy project page.

We then configure our PEAR installation for the specific application user and re-configure the bin and php library paths:

whitewashing@desktop:~$ pear create-config /home/whitewashing/ .pearrc
whitewashing@desktop:~$ pear config-set php_dir /home/whitewashing/whitewashing/trunk/vendor
whitewashing@desktop:~$ pear config-set bin_dir /home/whitewashing/whitewashing/trunk/bin

This configuration assumes that we will install all our stuff into our development trunk. From there also the PEAR installed libraries might be copied into branches or tags. PEAR Project tests, configuration and web-files will still be put by default under $HOME/pear/*. We don’t need them for our applications.

Now we install all the dependencies our project needs, in this case Zend Framework, Doctrine 2, HTML Purifier:

whitewashing@desktop:~$ pear channel-discover pear.zfcampus.org
whitewashing@desktop:~$ pear install zfcampus/zf-alpha
whitewashing@desktop:~$ pear channel-discover htmlpurifier.org
whitewashing@desktop:~$ pear install hp/HTMLPurifier
whitewashing@desktop:~$ pear channel-discover pear.phpdoctrine.org
whitewashing@desktop:~$ pear install pear.phpdoctrine.org/DoctrineORM-2.0.0

Now we have all three of the packages installed in our project folder whitewashing/trunk/vendor, see:

whitewashing@desktop:~$ ls -aFl whitewashing/trunk/vendor/
total 680
drwxr-xr-x  7 whitewashing www-data   4096 2009-12-13 14:45 ./
drwxr-xr-x  8 whitewashing www-data   4096 2009-12-13 14:36 ../
drwxr-xr-x  3 whitewashing www-data   4096 2009-12-13 14:43 .channels/
-rw-r--r--  1 whitewashing www-data     57 2009-12-13 14:45 .depdb
-rw-r--r--  1 whitewashing www-data      0 2009-12-13 14:45 .depdblock
drwxr-xr-x  5 whitewashing www-data   4096 2009-12-13 14:45 Doctrine/
-rw-r--r--  1 whitewashing www-data 582208 2009-12-13 14:45 .filemap
drwxr-xr-x 20 whitewashing www-data   4096 2009-12-13 14:39 HTMLPurifier/
-rw-r--r--  1 whitewashing www-data    629 2009-12-13 14:39 HTMLPurifier.autoload.php
-rw-r--r--  1 whitewashing www-data    274 2009-12-13 14:39 HTMLPurifier.auto.php
-rw-r--r--  1 whitewashing www-data    545 2009-12-13 14:39 HTMLPurifier.func.php
-rw-r--r--  1 whitewashing www-data   9299 2009-12-13 14:39 HTMLPurifier.includes.php
-rw-r--r--  1 whitewashing www-data    955 2009-12-13 14:39 HTMLPurifier.kses.php
-rw-r--r--  1 whitewashing www-data   8831 2009-12-13 14:39 HTMLPurifier.php
-rw-r--r--  1 whitewashing www-data  11901 2009-12-13 14:39 HTMLPurifier.safe-includes.php
-rw-r--r--  1 whitewashing www-data      0 2009-12-13 14:45 .lock
drwxr-xr-x  8 whitewashing www-data   4096 2009-12-13 14:43 .registry/
drwxr-xr-x 59 whitewashing www-data   4096 2009-12-13 14:36 Zend/
-rw-r--r--  1 whitewashing www-data  19537 2009-12-13 14:36 zf.php

And both Doctrine and ZF registered their binary CLi tools inside the whitewashing/trunk/bin/ folder:

whitewashing@desktop:~$ ls -aFl bin/
total 20
drwxr-xr-x 2 whitewashing www-data 4096 2009-12-13 14:45 ./
drwxr-xr-x 8 whitewashing www-data 4096 2009-12-13 14:36 ../
-rwxr-xr-x 1 whitewashing www-data   50 2009-12-13 14:45 doctrine*
-rwxr-xr-x 1 whitewashing www-data  169 2009-12-13 14:45 doctrine.php*
-rwxr-xr-x 1 whitewashing www-data 1511 2009-12-13 14:36 zf*

We now have the full control over the versions of our dependencies, we can call “pear upgrade ” whenever we want to update one of the ZF, Doctrine or HtmlPurifier libraries inside our application.

Now some magic is gonna happen, we start to develop our application and such which is all not really interesting for this topic. At some point we want to package it all up into a PHAR file and distribute it. We want to package our application in one big phar file. We also want to make sure that the configuration files in whitewashing/trunk/application/configs/ are not distributed, but have to be created on the server and are kept that way. We could write an installer script for this configuration management.

The reference for PHAR files is the PHP Manual for the Basics and Cal Evans’ two posts (1, 2) on this topic, aswell as a post on Geekmonkey. Contrary to most other PHP extensions, PHAR has an extensive documentation, however its not organized terribly well. Also there are no real use-cases and scenarios discussed, methods are only looked at in isolation. Cals posts are very good on understanding how to package up different libraries, but there is no word on distributing web applications. That is where the Geekmonkey post comes in to wire it all together.

For a Zend Framework application that should have both a web and a cli (cronjobs) entry point into the application we need a specific stub file for the PHAR bootstrapping. A stub is a little PHP script that is executed whenever your PHAR file is included into your php script. It is essentially a front-controller for your PHAR application. It also has mount capabilities that allow to import files or directories from outside into the PHAR context. This is a powerful feature that is required to distribute configurable applications like our blog.

This screenshot shows how the application is currently structured in development mode. In production its structure should look like:

whitewashing
|--application
|  |--configs
|     |-- my application config files are all here...
|--bin
|  |--whitewashing.php
|--public
|  |--index.php
|  |--.htaccess
|--whitewashing.phar

The whitewashing.php and index.php files are the application entry points that only include the phar file and trigger the application bootstrapping that will be included in the Stub file. They both look like:

<?php
define('EXTERNAL_APPLICATION_ROOT', __DIR__."/../");
include EXTERNAL_APPLICATION_ROOT."/whitewashing.phar";

Including a PHAR file essentially has two consequences:

• The PHAR path will be added to your include path.
• The stub file will be executed.

Our application stub looks like this:

<?php

if(defined('EXTERNAL_APPLICATION_ROOT')) {
    // Mount the external application/configs directory as config if it exists.
    if (file_exists(EXTERNAL_APPLICATION_ROOT."/application/configs")) {
        Phar::mount("application/configs", EXTERNAL_APPLICATION_ROOT."/application/configs");
    }
}

/** Zend_Loader_Autoloader */
require_once 'Zend/Loader/Autoloader.php';
$autoloader = Zend_Loader_Autoloader::getInstance();

if (php_sapi_name() == "cli") {
    require_once 'bin/whitewashing.php';
} else {
    require_once 'public/index.php';
}

__HALT_COMPILER();

The first bit of the stub mounts the external application configs directory into the stub and hides possible directories that are present at this location in the PHAR file. This allows us to distribute our application with a default configuration, but allows any user to replace the configuration files to fit the application to his need.

The second bit loads Zend Framework Autoloader that is required by the bootstrapping mechanism. The third bit decides whether this request is executed from the CLI- or the Web-Entry point of the application. The fourth bit, __HALT_COMPILER(); is a technically required call inside your stub-file.

Now that we have a stub-file for our application, we can package it and distribute it. I am using a modified version of Cal Evans example for this. I have extracted his directory traversal to find all the relevant into a re-usable FilterIterator implementation. I pasted my package.php a Gist on Github. Now this should probably be put into the build context of your application, possibly as a phing or ant task or something alike.

Now what this build process does not manage is the creation of the application entry point php and .htaccess files, but since they won’t ever change its easy to add them to the build directory for now. An even more sophisticated version of the build script would lead to the creation of an additional tar.gz of the complete application folder. Our deployment process would then be as easy as:

• If the application is not installed yet, unpack the tarball into its location.
• If the application should be updated, just replace the PHAR file.

If you need the ability to go back to any version of your application you could make use of symlinks.

PHP CodeSniffer Support for Netbeans

I dived into the code of my new favorite IDE Netbeans these last days and came up with an extension module, which adds PHP CodeSniffer Support on a per file basis to make my life much easier. It shows warnings and errors as annotations to the Editor and marks the affected lines in yellow and red.

|image0|

[STRIKEOUT:My Java skills being very bad, it will only work on Linux currently, since the PHP Code Sniffer “binary” is hardcoded into the Java Source code. You have to create a “/usr/bin/phpcs2” executable, which is a wrapper that looks like:]

With Manuals extensions (see the comments) the module now works without the wrapper script. It might even work under Windows now. Yet now the Zend Coding standard is enforced though. I am working on making that one configurable next.

You can install the NBM module install file from the GitHub repository into Netbeans and it “should” work then.

I hope to get more familiar with Netbeans in the future to add some more PHP tools and enhance Code Sniffer support.

Using Symfony Dependency Injection with Zend_Application

As a follow-up to my recent post on Dependency Injection containers and Zend_Application I was eager to find out if its possible to integrate the new Symfony Dependency Injection Container into the Zend Framework. To my surprise it’s possible without having to make any changes to one of the two components. An example use-case would look like:

$container = new sfServiceContainerBuilder();

$loader = new sfServiceContainerLoaderFileXml($container);
$loader->load(APPLICATION_PATH.'/config/objects.xml');

$application = new Zend_Application(
    APPLICATION_ENV,
    APPLICATION_PATH . '/config/application.xml'
);
$application->getBootstrap()->setContainer($container);
$application->bootstrap()
            ->run();

Resources instantiated by Zend_Application are then injected into the container by the name of the resource and are given the resource instance. Any object from the Symfony container can then use these dependencies in their object setup. The only drawback of the integration is the fact that the Symfony Container is case-sensitive in regards to the service names but Zend_Application lower-cases all service before injecting them into the container. The following code is a restatement of my previous example of a MyModel class which requires a Zend_Db and Zend_Cache constructor argument.

$container->register('myModel', 'MyModel')
          ->addArgument(new sfServiceReference('db'))
          ->addArgument(new sfServiceReference('cache'));

Access to a MyModel instance with its dependencies is then granted through the call $container->myModel throughout the application. Make sure to call this after running Zend_Application::bootstrap, so that the Resource dependencies are injected first.

PDT: Back to easyeclipse with PHP extension

I am disappointed by PDT for Eclipse. The current version strikes my rather old machine with “just” 512mb to death. Building projects is not possible even when assigning eclipse all my available memory, because of an Java Heap Error (See this blog post). This effectively hinders you to use all the completion and hint features, because PDT does not know about any functions, classes and their phpdoc descriptions.

PDT is also missing some very important features. Marking all occurrences of a function, class, constant or variable on click for example. PHP Errors and Warnings are display in a way so that you won’t find them and if you do, you won’t understand what they mean.

So I am going back to my easyeclipse with php combination, which always worked well, the only problem being that it is no longer under active development. At least it has all the features I need for now.

Introducing: Zend Controller Scaffolding

In the last couple of weeks I came across the idea of building a little component for scaffolding in Zend Framework. After playing around with this idea a little I came up with an Zend_Controller_Scaffolding object, which extends the Zend_Controller_Action object.

It takes any Zend_Db_Table_Abstract object and generates create and update forms using Zend_Form and displays them in the Controller. The component is very easy to use, as this example shows:

class SomeController extends WW_Controller_Scaffolding {
    public function init() {
        $this->setScaffolding(new ZendDbTableModel(), $options);
    }
}

You now have access to the following actions of the controller: some/create some/edit some/delete and some/index (some/list). They handle all your model editing dreams.

All you have to own is the WW_Controller_Scaffolding Library Class and a folder of scaffolding view templates both of which are bundled in an archive you can download.

Download

• Download Version 0.5.5

Changes

• 0.5.5: Put Component into own Namespace (WW = Whitewashing). Make it possible to hide fields in the list via the options. Allow to specify scaffolding view scripts folder via options.

Install & Usage

Untar the two folders include/ and views/ into your
Zend Framework project application directory.

Make sure the include folder is in your Zend_Loader
include path or move the Scaffolding.php so that it
is placed in your library include path.

Define each Controller that should be a Scaffolding Interface
as:

class SomeController extends WW_Controller_Scaffolding {
    public function init() {
        $this->setScaffolding(new ZendDbTableModel(), $options);
    }
}

Where $options is an array or Zend_Config object with any of the following keys:

  'allow_edit_primary_key' True/false - Whether the form allows you to set the
                          Primary Key fields or not

 'field_names' an Array of the Database Tables field names mapped to Label Names

 'hide_list_fields' Array of database table field names that should not be displayed in the list overview.

 'checkbox' an Array of Database Table field names that should be represented
            as Checkbox. Useful for TINYINT(1) fields that represent boolean decisions.

 'view_folder' Sometimes you want to use different views for scaffolding in one project. Use
               this variable and copy the scaffolding folder for each component you want to
               change the view basics for.

Todos & Problems:

• Many To Many Relationships are not implemented yet (Using MultiSelec).
• Compound Keys are probably not working correctly
• Relationships on non-primary key fields probably don’t work as expected
• Relationships with lots of data are not scaled down for easy administration.

Please report any bugs and feature requests or recommendations to kontakt at beberlei dot de.

My 2 cents on Zend_Search_Lucene Update Problems

Zend_Search_Lucene (or Lucene in general) does not support an update statement, therefore we must check via find() if a specific article that should be updated already exists.

Numerous solutions exist for this problem (for example this PDF Tutorial by a Zend Programmer). Mine looks as follows. The first thing is to Set an Analyzer other from the default one. The Default Analyzer only looks at letters, not at numbers.

Zend_Search_Lucene_Analysis_Analyzer::setDefault(new
    Zend_Search_Lucene_Analysis_Analyzer_Common_Utf8Num_CaseInsensitive()
);
$index = Zend_Search_Lucene::open('/path/to/index/directory');

We can now save an identifier containing letters and numbers, for example the md5 string of the Article Database ID.

$doc = new Zend_Search_Lucene_Document();
$field = Zend_Search_Lucene_Field::Keyword('id', md5($data['id']));
$doc->addField($field);
$index->addDocument($doc);

Before updating an entry you have to delete any entry that has been indexed before:

$hits = $index->find('id:'.md5($articleID));
foreach($hits AS $hit) {
    $index->delete($hit->id);
}

The Tutorial Talk on Search Lucene (given above) uses an extended version of this delete routine snippet that also checks if an article is up to date and does not have to be reindexed and deletes multiple entries of one article.

Integrated Zend_Layout into this blog

I finished with integrating Zend_Layout into this blog software today. At first I thought it is quite hard to use especially if you want to integrate dynamic navigational contents depending on the current module/controller/action setup. But I came up with a combination of actionStack and Named Response Sections as described in the ZF Documentation of the Layout component, which is quite easy to understand, use and extend.

At each of the major Controllers I put different actions from a specific Navigation Controller on the ActionStack (mostly global per Controller using the init() class method).

A typical controller looks like this:

class BlogController extends Zend_Controller_Action
{
    public function init()
    {
        $this->_helper->actionStack('tagcloud', 'Navigation');
        $this->_helper->actionStack('userinfo', 'Navigation');
    }
}

The NavigationController looks like this:

class NavigationController extends Zend_Controller_Action
{
    public function tagcloudAction()
    {
        // Do not render the content of this action to the default output.
        $this->getHelper("ViewRenderer")->setNoRender();

        [...]

        // Append content to secondaryNavigation named response section
        $this->getResponse()->append('secondaryNavigation', $this->view->render('navigation/tagcloud.phtml'));
    }
}

The layout.pthml then just calls certain named sections, for example <=? $this->layout()->secondaryNavigation; ?>

Using Zend_Http_Client to Ping Technorati

Send a ping to Technorati manually is a pain in the ass. Writing a little Method that handles this functionality for your own blog software is quite easy. Call the following method after you created or updated a post of yours:

private function _pingTechnorati()
{
    $xml = $this->view->render('ping_technorati.phtml');

    $httpclient = new Zend_Http_Client('http://rpc.technorati.com/rpc/ping');
            $httpclient->setHeaders('User-Agent', 'PHP/Zend Framework/Zend_Http');

    $httpclient->setRawData($xml, 'text/xml')->request('POST');

    $response = $httpclient->request();

    if($response->isSuccessful() === true) {
        return true;
    } else {
        return false;
    }
}

You also need a little view in XML format like it is described in the Technorati Ping Configuration Guide and ready to go you are!

Zend_Form - Rapid Development Plugin for Eclipse?

I found some time in the last weeks to reconsider Zend Framework, this time in the 1.5 version, for this blog software and rewrote all Forms using the new Zend_Form component. I have to say, i love it. Its very easy using the already existing Zend_Validate and Zend_Filter components as input for the generated form fields. Another plus is the handling of errors in input and re-entering/displaying text.

There is also a neat feature of the Zend_Form component, which allows to generate complete forms by passing an appropriately formatted Zend_Config_Ini object to the Zend_Form Constructor. So my idea was, why not program an Eclipse Plugin allowing to rapidly generate and edit forms using a point-and-click plugin window approach, which would ultimately generate your Form INI code.

This might have been a great Google Summer of Code idea for the Eclipse Foundation (since Zend is not taking part).

Implementing Zend Auth, Acl and Caching

I managed to work a bit on the blog, enabling me to have new instance in the series, How to refactor and extend your own blog software using Zend Framework. This time: Implementing meaningful Auth and ACL mechanisms and fixing view caching, which did not work in its original implementation due to different users groups sharing the same cached views.

In the last days I came across two in depth tutorials on Zend_Acl and Zend_Auth integration to MVC. Most people probably saw the DevZone article “Zend_Acl and MVC Integration (part 1)” by Aldemar Bernal on the Frameworks Frontpage. Another good article was written by Frank Ruske in the latest german PHP Magazin (Zipped Source Code of the Example). I took the best ideas of both articles and merged them into the existing components of my blog.

This now enables me to cache the site depending on the Auth status of the page user agent. The blog is now caching all views that are generated for guest users. Since there is no “registered” or “member” account yet, this means the blogs content is cached and served from cache for everyone except me when I am logged in. To get this work I added some simple additional check in Matthews Cache View Controller Plugin:

public function dispatchLoopStartup(Zend_Controller_Request_Abstract $request)
{
    $auth = Zend_Auth::getInstance();
    if($auth->hasIdentity()) {
        self::$doNotCache = true;
        return;
    }
    [..]
}

This prevents caching for registered identities.

Exim4 and Virtual Users .forward Files

Some days ago I began to wonder why some mails in mail account were filtered by Exims .forward file why some are not. It took until today that I found out what was the problem and how to fix it.

I implemented a virtual users/domains system for Exim using several different documentations and tutorials. Right now some parts of my system include the documentation on

http://www.tty1.net/virtual_domains_en.html

Now the problem is that the userforward router is only working for accounts whose $local_part is an existing system user. I came up with the following additional router that solves the problem. Its written for the MySQL table layout that is specified in the Tutorial above.

virtualuserforward:
  debug_print = "R: virtualuserforward for $local_part@$domain"
  driver = redirect
  domains = +local_domains
  file = ${lookup mysql{SELECT CONCAT(home,'.forward') AS forward FROM users \
        WHERE id='${quote_mysql:$local_part}@${quote_mysql:$domain}'}}
  no_verify
  no_expn
  check_ancestor
  directory_transport = address_directory
  file_transport = address_file
  pipe_transport = address_pipe
  reply_transport = address_reply
  allow_filter

  user = ${lookup mysql{SELECT uid FROM users \
        WHERE id='${quote_mysql:$local_part}@${quote_mysql:$domain}'}}
  group = ${lookup mysql{SELECT gid FROM users \
        WHERE id = '${quote_mysql:$local_part}@${quote_mysql:$domain}'}}

Now each virtual user is using the .forward file in its actual system user account home directory. The next problem was: $home is not defined in this case in the .forward file syntax: Its empty. So replacing $home with its obvious path leads to the final solution.

Multimediatreff on Frameworks

My neighbour gladly called my attention to the Multimediatreff, a relaxed conference that is held about three times a year, this last saturday on the topic: Frameworks and Content-Management-Systems.

It was held in a very nice conference center in cologne, and from what I heard about 200 people attended. The first two topics were on Joomla! and barrier-free Typo3. Since I am more the framework guy, I did not enjoy them too much. At least I got some input on how a barrier-free website has to function. The Joomla! presentation supported my prejudice, that programming in and for Joomla! produces ugly code.

I did like the presentations on frameworks, the most interesting being the CakePHP presentation by Timo Derstappen, since I haven’t looked into Cake for about one and a half years. I have seen that with the new to come version 2, the framework became even better, now supporting easy code generation and flexible site generation. But its still somehow a Ruby clone and forces me too much in my programming style.

The Zend Framework presentation was done by Carsten Möhrke, who also wrote the first german ZF Book. He showed the creation of a blog using Zend Framework, being the Hello World example for frameworks. He specifically showed, that you can use anything you want as a model, a Zend_Db_Table object or any other object. He also showed something like a Data-Mapper pattern in his application. I got some ideas from this I probably will write about some later time.

I also enjoyed the Rails presentation, because I never got the chance to look the Ruby guys at work of the shoulder. But this is even worse than CakePHP was my first observation. I am no fan of ActiveRecord being forced upon you as Model implementation.

The pizza battle, mysteriously announced by the host, was another highlight, as well as the drinks and snacks in between. All in all I can highly recommend the Multimediatreff for the next time.

ZF: Caching Pages via Front Controller Plugin

Today I implemented caching using a front controller plugin (see Matthews post on Zend Devzone) into the Whitewashing blog software via Zend_Cache and the Filesystem Backend. I ran some superficial Apache Benchmark tests to have a look at the gain change in performance.

First of all i used the complete Plugin from Matthews article on Zend Devzone and a missing function getCache() in the code fragment:

public function getCache()
{
    if( ($response = $this->cache->load($this->key)) != false) {
        return $response;
    }
    return false;
}

I loaded the Plugin into the front controller and initialized it with a Zend_Config_Ini object. This all works very smoothly and caching is ready to begin.

Testing came to the result that with 1000 requests, 10 of them concurrent (ab -n 1000 -c 10), which I know is a rather unrealistic assumption for this blog, the request time dropped by half, from six to three seconds (still lot of time, but this isn’t the best webserver).

Requests per second:    1.49 [#/sec] (mean)
Time per request:       6695.213 [ms] (mean)

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       26   30   4.0     30      43
Processing:  1556 6452 3372.1   5511   14768
Waiting:     1436 6129 3174.1   5145   14595
Total:       1583 6483 3373.2   5538   14810

Requests per second:    3.03 [#/sec] (mean)
Time per request:       3304.534 [ms] (mean)

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       26   31   6.3     30      73
Processing:   781 3214 1757.7   3103    7445
Waiting:      650 3055 1731.4   2928    7190
Total:        809 3245 1757.8   3135    7475

On Frameworks and Javascript Coupling

Zend announced they work together with the Dojo Team to integrate Javascript support into the Zend Framework. What this actually means is that you can write your Javascript Code in PHP if you’re javascript library of choice is Dojo.

Since Dojo is not my library of choice I would have to write a JQuery View Helper on my own or wait until somebody else releases one.

But why not write javascript? Its very easy with all those libraries and addresses most cross browser incompatibilities: One Django, one Zend Framework developer argue framework and javascript coupling is not the way to go based on the arguments that its breaks the MVC pattern, leads to function calls equal to those in javascript in its framework respective language, and that javascript is actually something any good webdeveloper should be able to program, even when its just using library components.

Using JQuery without any help from tools for about a year now I can say there are still those days where javascript drives me mad, but most of the time I its working perfect. Since Ajax related calls like GET, POST, and FORM stuff are oneliners I don’t see why one would need tighter integration of JS and frameworks.

Larger Javascript components like Popup Calendars, Autocomplete and the like are harder to integrate than in Rails or cakePHP, but thinking about the problem a little longer leads to powerful reusable solutions, that save time on integration after the first initial setup.

Simplifying ZF: Scope of Variables in View and Controller

As a follow up on the Zend Framework, “Web 2.0 Framework” My Ass! article referenced earlier I came up with some simplifications of the Zend View and Controller variable coupling.

Using some code in the article I created a new Zend_View_Extended class which offers the possibility to circumvent the access of variables via $this and directly registers each variable key of the Zend_View object as an own variable in the script template:

class Zend_View_Extended extends Zend_View_Abstract
{
    protected function _run()
    {
        while( list( $k, $v ) = each( $this ) ) ${$k} = $v;

        include func_get_arg(0);
    }
}

Now rather than calling $this->data you can call $data in a script template. Of course this comes with additional overhead of each variable being registered twice. I don’t know how this handles performance wise, but maybe unsetting $this variables after copying solves this. Also you have to overwrite the initView() method of your base Controller Action class.

Another simplification would be to allow for the following direct variable settings in any Controller Action, which would shorten the $this->view->variable call, to derive the same functionality via $this->variable. I haven’t tested this though.

class Zend_Controller_SimpleAction extends Zend_Controller_Action
{
    function __set($name, $value)
    {
        $this->view->{$name} = $value;
    }
}

Teaching Zend_Form some MVC

Lots of people complain that Zend Form runs counter to the MVC spirit, because it handles validation, business logic and view elements all in one acting as Model, Controller and View.

Extending the Zend Form component to handle all this different aspects in different layers of the application is rather easy though. What we want of a MVC compatible Form object is the following:

• The model escapes and validates all the data that is put into the form.
• The view decides on how the form is displayed.
• The controller moves data from the model to the view and back, handling the stages of the form request.

The first step is, allowing any Model that implements Zend_Validator_Interface to hook into the Zend_Form Validation process. We generate a new class, WW_Form_Mvc and allow a function setModel() to insert any Model object that implements Zend_Validator_Interface into the form. We extend isValid(), getMessages() and getErrors() to not only check all the form elements validators, but also the models validators. Please note that the array_merge() solution is not the correct way of how this snippet should work. Any merge operation of the messages and errors has to be on a field key level, which is not currently done.

class WW_Form_Mvc extends Zend_Form
{
    protected $model = null;

    /**
     * Extends isValid() method of Zend Form to check for validity of specified model
     * @param Array $data
     * @return Boolean
     */
    public function isValid($data)
    {
        $valid = parent::isValid($data);

        if($valid == true && !is_null($model)) {
            $valid = $this->model->isValid($data) && $valid;
        }
        return $valid;
    }

    /**
     * Extends getMessages() Validator Interface implementation of Zend Form to also
     * return the messages of the Model validation.
     * @return Array
     */
    public function getMessages($name = null, $suppressArrayNotation = false)
    {
        $messages = parent::getMessages($name, $suppressArrayNotation);

        if(!is_null($model)) {
             $form_messages = $this->model->getMessages();

             $messages = array_merge($messages, $form_messages);
        }

        return $messages;
    }

    /**
     * Extends getErrors() Validator Interface implementation of Zend Form to also
     * return the errors of the Model validation.
     * @return Array
     */
    public function getErrors($name = null)
    {
         $messages = parent::getErrors($name);

        if(!is_null($model)) {
             $form_messages = $this->model->getErrors();

             $messages = array_merge($messages, $form_messages);
        }

        return $messages;
    }

    /**
     * Set a Model object, which has to implement Zend_Validate_Interface
     * @throws Zend_Exception
     */
    public function setModel($model)
    {
        if($model instanceof Zend_Validate_Interface) {
            $this->model = $model;
        } else {
             throw new Zend_Exception('WW_Form_Mvc expects a model of type Zend_Validate_Interface');
        }
    }
}

We then extend the WW_Form_Mvc class to disable the automatic loading of decorators in the Constructor and additionally allow to pass a Model to the constructor as second argument:

class WW_Form_Mvc
{
    protected $model = null;

    /**
     * this overrides the original Zend_Form Constructor and skips
     * the decorator initialisation, because this is now being handled
     * by View Helpers
     */
    public function __construct($options=null, $model=null)
    {
        if (is_array($options)) {
            $this->setOptions($options);
        } elseif ($options instanceof Zend_Config) {
            $this->setConfig($options);
        }

        // Extensions...
        $this->init();

        if(!is_null($model)) {
             $this->setModel($model);
        }
    }

    // All the other stuff here
}

In our views we want to use helper methods to manage the displaying of the form. For each different style of form displaying, we can generate different helpers. For example a helper that would only apply the default decorators would look like this:

class WW_View_Helper_FormDefault
{
    /**
     * Load only default decorators on this Zend_Form object
     *
     * @param Zend_Form $form
     */
    public function formDefault(Zend_Form $form)
    {
        if($form instanceof Zend_Form) {
            $form->loadDefaultDecorators();
            return $form;
        }
    }
}

We can now use this helper and in any template say: <?= $this->formDefault($this->someForm); ?> We can now look at our controller action that implements this form and we will see that it does not look different from what we would have done before:

function formAction()
{
    $model = new SomeModel();

    $form = new WW_Form_Mvc();
    $form->setModel($model);

    // generate form here, adding elements and stuff

    if($form->isValid($_POST)) {
        $model->insert($form->getValues());
        $this->view->form = "Form was submitted!";
    } else {
        $this->view->form = $form;
    }
}

Isnt that nice? Now each part of the equation is doing what its supposed to do.

Overwrite ezcMvcController - A bit more rapid

ezComponents gets Mvc in its 2008.2 version. I have played around a bit with the alpha version, since I am currently searching for a good framework for a high performance application. My first benchmarks on ezcMvc just say: wh000pie! A lot faster as compared to the Zend Framework. Still ezcMvc is VERY loosely coupled and you have to write lots of lines to get where ZF gets you with less (more magic involved). As you can see from the tutorial on the ezcMvcTools component it is currently a bit unwieldily to work with ezcMvcController since you have to create and return result objects everywhere. This is very nice since it abstracts from the actual response type (could be www, mail, cli, anything). I have created a very little extension of the ezcMvcController class that hopefully serves you quite some time. You can append variables to the ezcMvcResult object by calling the magic __get and __set on the controller. Plus it offers a method to use the ezcMvcInternalRedirect instead of the result. See for yourself:

class myController extends ezcMvcController
{
    protected $result;

    public function createResult()
    {
        $actionMethod = $this->createActionMethodName();

        if ( method_exists( $this, $actionMethod ) ) {
            $status = $this->$actionMethod();
            if($status != 0) {
                $this->getResult()->status = $status;
            }
            return $this->getResult();
        } else {
            throw new ezcMvcActionNotFoundException( $this->action );
        }
    }

    protected function _redirect($uri)
    {
        $request = clone $this->request;
        $request->uri = $uri;
        $this->result = new ezcMvcInternalRedirect($request);
    }

    public function __get($name)
    {
        if(isset($this->getResult()->variables[$name])) {
            return $this->getResult()->variables[$name];
        }
        return null;
    }

    public function __set($name, $value)
    {
        $this->getResult()->variables[$name] = $value;
    }

    public function __isset($name)
    {
        return isset($this->getResult()->variables[$name]);
    }

    protected function getResult()
    {
        if($this->result === null) {
            $this->result = new ezcMvcResult();
        }
        return $this->result;
    }
}

You can now use a controller in the following way:

class dashboardController extends myController
{
    public function doIndex()
    {
        $this->cookie = "Cookie!"; // Proxy to $ezcMvcResult->variables['cookie']
    }

    public function doRedirect()
    {
        $this->_redirect("/");
    }
}

Very nice! The next thing I have to extend in ezcMvc is automagical matching of controller and action names to view output names with a special View Handler that takes care of this. This saves another bunch of work you have to cope with in the current standard setup.

Next ZF Mini Release: 1.7.1

In the next days the next mini release of the Zend Framework will be released and I want to keep you up to date of my progress on it. I got lots of feedback on the jQuery component and also had the possibility to fix some bugs and documentation issues. I also fixed some minor bugs that have been in the Zend_Soap_Wsdl stuff i added for the 1.7 release. And a question for the 1.8 release that is due around February 2009: What do you think is missing in the jQuery support to ZF? Any widgets or stuff that you want to have included?

ZF 1.7: jQuery is in! Zend_Soap with lots of Bugfixes.

The next version of Zend Framework will appear quite soon (16th November) and there will be lots of buzz about Zend Amf, the new component sponsored by Adobe to support Adobe Flash (or other tools) to PHP communications.

Besides the buzz there is some stuff from me getting into the Zend Framework. jQuery View and Form Helpers will allow you to throw the Dojo stuff away and use the great jQuery. This is of course inherently subjective, but still different opportunities are always great.

Additionally I have been fixing quite some bugs on the Zend Soap component and will manage to fix further stuff until the 1.7 release I hope. This will make especially Wsdl and AutoDiscover produce valid WSDL XML plus adds setObject() support to Zend_Soap_Server.

You can now choose between several strategies to evaluate complex types in WSDL Autodiscovering. Array of Datatypes (simple or complex) via type[] syntax and complex objects can be parsed differently now based on settings.

Performance in Calypso (and Bugs)

I have tested Calypso performance some weeks ago and have not been able to write on it before. Don’t use it please its fucking slow and cannot be optimized in the current architecture, because the template will NOT be parsed into intermediate PHP code.

Additionally there are some bugs:

1. You cannot currently use UTF8 encoding since htmlentities deep in the escape mechanism cannot be given the utf-8 specification.
2. Triple or higher inheritance may not work under some circumstances. Which sucks.

I sadly have no time currently to fix any of the issues poping up with Calypso, so please don’t use it for anything production.

Encouraging forks for Calypso DTL

I still get a bunch of emails regarding the Calypso DTL template engine that I have written some month ago and which i can currently not maintain any further (see my posting on that). Still I encourage anyone to fork the project and develop it further. Personally what I think is missing are some bugfixes and enhancements regarding the compiling and parsing. I created a GITHub Project for Calypso DTL, so collaborative development is very easy and added general TODOS to the README.txt that is on the front page of github project page.

Updating from KDE3.5 to KDE4 with Mac-Style Menubars

It seems KDE4 has no Mac-Style menubars and when you upgrade from 3.5 to 4.1 within the same config folder (say you use the kubuntu distribution upgrade installer like I did) you end up with no menubars at all. To solve this little issue find a variable “macStyle = true” in the /home/username/.kde/share/config/kdeglobals file and switch it to macStyle = false and you have your menubars back.

DojoX Django Template Language Example with Data from PHP/JSON

This is a simple example how you would use DojoX Django Template Language component filled with data from a JSON GET Response:

<script type="text/javascript" src="/development/dojo-release-1.1.1/dojo/dojo.js" djConfig="parseOnLoad:true, isDebug:true"></script>

<script language="javascript" type="text/javascript"><!--
dojo.require("dojox.dtl");
dojo.require("dojox.dtl.Context");
dojo.require("dojox.dtl.ext-dojo.NodeList");
dojo.addOnLoad(
    dojo.xhrGet({url: "dojotest.php", handleAs: "json", handle: function(data,ioArgs){
        if(typeof data == "error"){
            console.log("error?",data);
        } else {
            dojo.query("#test").dtl("I am eating {{food.fruit}} and {{food.meat}}", data);
        }
    }
}));
--></script>

::

    <?php
    header('Content-type: text/json');
    $food = array('food' => array('fruit' => 'apple', 'meat' => 'chicken'));
    echo json_encode($food);
    ?>

Wrestling with Dojo

I have to complain about the Dojo Toolkit (Considering the project I am using this for). It has the worst documentation and API guides I have ever seen. The documentation (called the Dojo Book) has an unnecessary complex structure to click through and its contents are wrapped around examples that are so overly complex that you click away the pages when they get smashed at your head.

The API too, has no clear structure and hides its contents behind cryptic names such as “dnd”, “fx”, “rpc”. Descriptions of the functions sum to very small sentences and additional options are listed without any description at all other than a link to where they are defined in the project. There are no short examples given to any function like jQuery does.

In general I think the Dojos Site is too complex and hard to navigate. The layout seems to change massively on each click and you it is hard recognize common patterns. Dear Dojo Team, please make your project more accessible!

First impressions on Zend_Soap and a basic implementation

The Zend Framework release candidate for version 1.6 includes a new component for SOAP operations. Zend_Soap_Server/Client extend the PHP functionality of the SOAPClient and SOAPServer objects, which by itself is trivial. The more important functionality it ads to the package is the AutoDiscovery Component.

Generally you can use SOAP in the so called “non-wsdl” mode, that is if you specify the correct options like the soap service location and uri, you don’t need any description of the service for the clients. This is only useful if you’re using the SOAP Service internally since you then know about all the available functions and methods. If you want to offer the Service for external users you want to use the WSDL-mode: Generate a WSDL that describes the services available methods, their parameters and return types is an important task.

Using Zend_Soap_AutoDiscover you can generate your WSDL file automatically by reflecting on the given Service Class methods. This works as follows. We setup a simple service:

class HelloWorldService
{
    /**
     * @return string
     */
    public function helloWorld()
    {
        return "Hello World!";
    }

    /**
     * @return array
     */
    public function getFruits()
    {
        return array('apple', 'orange', 'banana');
    }
}

It is important that you specify the Doc Comments @param and @return otherwise the AutoDiscovery of the correct parameter and return types cannot be resolved. We will now setup a simple SOAP Server access point, that will also generate our WSDL file for description of this HelloWorld service:

require_once "HelloWorldService.php";
require_once "Zend/Soap/Server.php";
require_once "Zend/Soap/AutoDiscover.php";

if(isset($_GET['wsdl'])) {
    $autodiscover = new Zend_Soap_AutoDiscover();
    $autodiscover->setClass('HelloWorldService');
    $autodiscover->handle();
} else {

    $soap = new Zend_Soap_Server("http://localhost/soapserver.php?wsdl"); // this current file here
    $soap->setClass('HelloWorldService');
    $soap->handle();
}

Now we have our SOAP Service up and running and any client can access the HelloWorldService class from a remote or local location with just this simple lines:

require_once "Zend/Exception.php";
require_once "Zend/Soap/Client.php";

try {
    $client = new Zend_Soap_Client("http://localhost/soapserver.php?wsdl"); // Servers WSDL Location
    $string =  $client->helloWorld();
    $fruits = $client->getFruits();

    var_dump($string);
    var_dump($fruits);
} catch(Zend_Exception $e) {
    echo $e->getMessage();
}

This is easy. Additionally Zend_Soap offers a WSDL class to generate your own WSDL file based on your special preferences, which is a nice feature. I guess only some people can write a correct WSDL XML specification file on their own from scratch, so using a powerful helper is reasonable.

The usage of Zend_Soap is quite simple and straightforward as is PHP5’s internal SOAP Service and might therefore gain widespread use. I have never tested PEAR’s WSDL Autodiscovery, so i cannot draw comparisons.

Zend Helper for jQuery UI Widgets Demo

I finished a pretty demo for all the jQuery UI widgets view helpers for my Zend Framework proposal. It utilizes the jQuery View helper which is further down the proposal road. You can have a look at how nicely the Javascript is generated for the different jQuery UI Widgets. A further demo showing functionality of the form decorators and elements will follow when I finished these components.

Update: I also added a demo showing the form elements usage I have just finished in their first versions. I committed the classes to the SVN so you can already check them if you want.

Benchmark and Optimize Zend Framework Performance

In the fall of Rasmus presentation (Simple is Hard) on FrOSCon I tried to optimize Zend Frameworks performance a little bit. There has also been a little discussion on the Zend Framework Mailing List on this topic.

I haven’t changed great parts of the code or anything, I just benchmarked how different global include strategies affect the overall performance of my blog software written with help of the ZF. The following include strategies were tested (I’ve used the Zend Framework 1.6 RC2 package for all of them):

1. Zend_Loader Autoload, Default PHP Include Path
2. Zend_Loader Autoload, Swapped Include Path
3. ZF 1.6 RC2 stripped from all “require_once” dependencies, Zend_Loader Autoload, Swapped Include Path
4. ZF 1.6 RC2 stripped from all “require_once” dependencies, no autoload, used inclued to find file dependencies and require (not _once) them all on startup.

To strip all the require_once from the Zend Framework source code, i built a little script to do that for me. For the last test I wrote a little script that used the inclued_get_data() function to built a correct dependency tree for all includes. I have put each configuration of my Zend Framework install 30 seconds under siege with 5 concurrent requests. I have rerun all tests with APC and without APC.

ZendX_JQuery is Ready for Recommendation

Yesterday night I finished the proposal for ZendX_JQuery, a View Helper and an Ajax Helper hopefully to be included in the Zend Framework Extras Library in the next release after 1.6. I moved the proposal to the “Ready for Recommendation” section now, so it can be reviewed a last time by the community and by the Zend people.

For those people already using an earlier version, i have squashed about 4-5 bugs I found writing the Unit-Tests for this component. The test coverage for both helpers is 100% although the jQLink Helper might need some additional Use-Case scenarios to ensure proper working. You can find the current version at my SVN repository.

Import and PYTHONPATH Problems with PyDev and Linux

Today is began to configure my PyDev Eclipse environment since I want to learn Python and need it for a programming aspect of my forthcoming diploma thesis. I ran into several difficulties getting the plugins run dialog to work. It seems that configuring the python interpreter under linux and adding all the correct include paths is not enough, even when following the PyDev FAQ.

Nearly giving up I came up with sort of a hack to get it running. Eclipse can execute arbitrary “External Tools” (Find it at: “Run -> External Tools -> Open External Tools Dioalog”). I used this facility to integrate an external tool “Python” with path “/usr/bin/python”. Now to automatically execute the currently active python script, you can specify the line: “${resource_loc}” in the arguments field of the dialog and you’re good to go.

You now circumvented the PyDev internal execution scheme and work with the pure command line interpreter. This way all the problems of PyDev with the PythonPath, Symlinks and whatever are gone.

Update: Another note maybe, you can also add the action “running the last used external tool again” as keyboard shortcut in the Eclipse preferences. This simplifies the execution even more.

Complete Django Template Engine Implementation for PHP5 - Downloadable Now

I prepared a downloadable version of my Django Template Language Engine Clone for PHP5 and the Zend Framework (although it can easily be used Standalone). It also implements two helpers that aid in rapid ajax/Dojox deployment. You can easily specify template blocks that should be rendered by Javascript via DojoX DTL, which are then marked, compiled and shown by the client using JSON data from the server. I included a demo to show this feature.

You can grab the Tarball and there is also a README with some information. I would appreciate if anybody would test the lib and gave some feedback.

I will soon start to push this component in the ZF proposal process. I still have to finish the class structure before I can move to the Ready for Review stage. The library will max end up in Zend Extras. Therefore I will have to say again: This View Renderer shall be no replacement for the current Zend View. It is only an alternative which allows for some neat functionality and stricter View / Controller logic separation.

UPDATE: I finished a little page for what I call the Calypso DTL. See it here: Calypso Site. I have put up a little tutorial how to get the Template Engine running with Zend Framework and as a standalone component.

Greaat, Python can control your Wiimote

In the last days I had lots of time to play with my Wiimote and its bluetooth capabilities. Sadly there is actively supported driver in non-C (and I am not the biggest C programmer). I found this little prototype driver written in Python by Will Woods which is using pyBluez to communicate with my wiimote and have extended it quite a bit. WiiLinux is a huge resource that offers information on reverse engineering of the Wiimote protocol.

The driver is still far from finished or usable outside some demo applications, but that might still be included in the future. You can download this little goodie and have a look at it.

Constructions...

Some people complained: Yes comments do not work YET, and also the links behind categories and Tags show nothing. Everything else follows shortly. I am programming this whole blog using the Zend Framework to have a bit of learning experience. It will take some more time than if I would do it in “the old ways”.

Helpers in Zend_View: Class Prefix

I have to say, the Zend Framework documentation is a bit imprecise sometimes. Regarding custom Helpers in Zend_View it states:

The class name must, at the very minimum, end with the helper name itself, using CamelCaps. E.g., if you were writing a helper called “specialPurpose”, the class name would minimally need to be “SpecialPurpose”. You may, and should, give the class name a prefix, and it is recommended that you use ‘View_Helper’ as part of that prefix: “My_View_Helper_SpecialPurpose”. (You will need to pass in the prefix, with or without the trailing underscore, to addHelperPath() or setHelperPath()).

What does that mean regarding how your Helper has to be called? The default prefix for a Helper function that should be called foobar() is actually:

class Zend_View_Helper_Foobar()
{
    function foobar() { }
}

It took me some time to realize that. Why doesn’t Zend just give an example like that? To change the prefix in a context of the Zend_Controller_Action you have to do something like this:

class SomeController extends Zend_Controller_Action
{
    function init() {
        $this->initView();
        $this->view->addHelperPath("pathtohelpers", "ClassPrefix");
    }
}

A knight’s tour solver in pascal

My professor has a task on the current problem set of my pascal programming course to program, demanding a recursive program finding a solution to the knights tour problem on a chess board of size n>=3 given a starting position. I used a backtracking algorithm (at least that is what wikipedia tells me it is), by trying all possible combinations and setting steps back, when they do not lead to a solution.

Beginning with n = 7 my algorithms gets very slow, caused by the backtracking algorithm which is not very efficient. I found some interesting articles regarding this problem using graph theory to solve it. Some pretty big minds have thought about the problem, I guess its not for me to find such an solution for a little task of the weekly problem set.

• Wolfram MathWorld: Knights Tour
• Presentation Knights Tour (German Language)

Knights Tour using Warndorff’s rule

When using Warndorff’s rule for solving the Knights Tour problem, the solution is found within a second. It took some time for me to realize it in PASCAL though using a RECORD type to match the jumps to the number of free fields and then sort them. You can view my source code here.

A knight’s tour solver in pascal - supplement

Following my earlier post on the Knights tour: It seems it is crucial for a backtracking algorithm what the knights starting position is, and even more important in what order the jumps are tried. Because there are 8 possible moves for a knight, you have to decide in which order they should be performed. If there is not a deviation when backtracking the algorithm gets stuck in almost the same positions very very often.

I rewrote the order of jumps for my 7x7 board and found a solution in seconds. Using the same order for an 8x8 board gets me stuck in endless tries again. I see its important to implement the Warnsdorff’s algorithm, perhaps I will do so.

Linux style MySQL Shell prompt

Just add the following to the [mysql] section in your .my.cnf file:

prompt=\u@\h [\d]>\

Writing a Game for KDE4

I am writing PHP / Web Applications for about 8 years now, but I am not so much the pro in writing desktop applications. I know little or no C/C++/Java, but want to change that soon. Since I use KDE for both my desktop and laptop it seems reasonable to write a KDE 4 application.

What about ideas? I wanted to write a game for so long and in the last years my girlfriend and I tried lots of 2 person player games. One of them I wanted to re-create as a computer game.

The possibilities are:

• The Game of the Amazons
• Moruba - Oware or a variant.

All 3 games are time intensive and very strategic. Moruba can be played for lots of hours (My gf and I played one game for about 5-6 hours). Its probably more easy to write algorithms for the computer player of Oware or Moruba than for Amazons, but that should not affect my decision. I will decide in the next weeks, but I should probably concentrate on my semesters final exams in the first place.

Update: Moruba seems to have different rules than the game we are playing. I will elaborate on that soon.

Zend_View Hack for implementation of RSS Feed

Using Zend_View for your site and want to implement an RSS Feed? You might run into the problem that the XML header definition is interpreted as PHP code:

<?xml version="1.0" encoding="ISO-8859-1"?>

Before you start cursing the world try this (rather obvious) workaround:

<?php echo '<?xml version="1.0" encoding="ISO-8859-1"?>'; ?>

Using a self defined Zend_Loader

When you start writing with a Framework like Zend like I did in the last weeks, you probably are still haunted by all the bad habits of pseudo object orientated PHP design, using lots of require and include’s. Zend_Loader doesn’t seem to help at first, its just another command (Zend_Loader::loadClass or ::loadFile). It sucks to register lots of autoload paths too.

I wrote my own Zend_Loader child class. It grabs all include paths from a config File using the Zend_Config class, and completely frees you of thinking about includes.

class WWLoader extends Zend_Loader
{
    private static $dirs = NULL;

    public static function loadClass($class)
    {
        if(self::$dirs===NULL) {
            //
            // Include directories of this application are saved in a configuration file. If it has not been
            // loaded before do so now and safe everything to the private static variable $dirs which will
            // then be used in further loadings
            //
            $dirs = array();
            $conf_app_path = new Zend_Config_Ini(sprintf('%s%s', constant('ZEND_CONFIG_PATH'), 'application.ini'), 'appincludepath');
            self::extractPaths($conf_app_path->toArray(), $dirs);
            self::$dirs = $dirs;
        }

        parent::loadClass($class, self::$dirs, true);
    }

    public static function autoload($class)
    {
        try {
            self::loadClass($class);
            return $class;
        } catch (Exception $e) {
            return false;
        }
    }

    /**
     * Given an array with subkeys of include paths this function unifies this array to
     * a single one and returns the result in the second argument $dirs which is given
     * by reference.
     *
     * @param Array $array
     * @param Array $dirs
     */
    public static function extractPaths($array, &$dirs)
    {
        foreach($array AS $k => $v) {
            if(is_array($v)) {
                self::extractPaths($v, $dirs);
            } else {
                $dirs[] = $v;
            }
        }
    }
}

ZEND_CONFIG_PATH is the only constant I use in my application. I initialize the following at the beginning of my bootstrap file:

define('ZEND_CONFIG_PATH', dirname(__FILE__)."/../application/config/");

require_once 'Zend/Loader.php';
require_once dirname(__FILE__).'/../application/include/WWLoader.php';
Zend_Loader::registerAutoload();
Zend_Loader::registerAutoload('WWLoader');

After that, its just Objects.

ZF: Managing 404 Errors with Version 1.0.3

Ok, the description of the Link referring to the version 0.9 solution for Managing 404 errors due to missing modules or actions is wrong. It is not enough to replace Zend::loadClass with Zend_Loader::loadClass. Other functions have been renamed, my current snippet is:

/**
 * Original Snippet from: http://www.bigroom.co.uk/blog/managing-404-errors-in-the-zend-framework
 */
    class NoroutePlugin extends Zend_Controller_Plugin_Abstract
    {
        public function preDispatch(Zend_Controller_Request_Abstract $request )
        {
            $dispatcher = Zend_Controller_Front::getInstance()->getDispatcher();

            $controllerName = $request->getControllerName();
            if (empty($controllerName)) {
                $controllerName = $dispatcher->getDefaultControllerClass($request);
            }
            $className = $dispatcher->formatControllerName($controllerName);
            if ($className)
            {
                try
                {
                    // if this fails, an exception will be thrown and
                    // caught below, indicating that the class can't
                    // be loaded.
                    Zend_Loader::loadClass($className, $dispatcher->getControllerDirectory());
                    $actionName = $request->getActionName();
                    if (empty($actionName)) {
                        $actionName = $dispatcher->getDefaultAction();
                    }
                    $methodName = $dispatcher->formatActionName($actionName);

                    $class = new ReflectionClass( $className );
                    if( $class->hasMethod( $methodName ) )
                    {
                        // all is well - exit now
                        return;
                    }
                }
                catch (Zend_Exception $e)
                {
                    // Couldn't load the class. No need to act yet,
                    // just catch the exception and fall out of the
                    // if
                }
            }

            // we only arrive here if can't find controller or action
            $request->setControllerName( 'blog' );
            $request->setActionName( 'noroute' );
            $request->setDispatched( false );
        }
    }

I hope someone needs this as much as me.

Complaining about Zend_Filter_Input

Zend_Filter_Input is really nice to be sure form data is filtered correctly, but i have some serious complaints: Why the hell are the error messages so user-unfriendly? I really would like to use the output of Zend_Filter_Input::getMessages(), but I could never trust to show them to a user of my website: They are obviously written for developers. Some examples:

‘’ does not appear to be an integer ‘’ is an empty string

I use a ton of validators to filter the comments form and all possible error messages that can be thrown are not “user-save”. Because all these messages are handled in each Validator class its almost impossible to change them without going nuts:

$validators = array(
    'article_id' => array(
        'Int',
        new Zend_Validate_Int(),
        array('GreaterThan', 0)
        ),
    'username' => 'Alnum',
    'userEmail' => new Zend_Validate_EmailAddress(Zend_Validate_Hostname::ALLOW_DNS | Zend_Validate_Hostname::ALLOW_LOCAL, true),
    'cp'   => array(
        'Digits',                // string
        new Zend_Validate_Int(), // object instance
        array('Between', 1138, 1138)  // string with constructor arguments
    ),
    'comment' => 'NotEmpty',
);

This simple $validators requirement creates templates for error messages in each validator object, that are: Zend_Validate_Int, Zend_Validate_Alnum, Zend_Validate_EmailAddress, Zend_Validate_Hostname, and the not empty message. To allow for user friendly messages one had to make child objects of each of the Validators and edit the error messages accordingly. I avoid this by printing my own errors from outside the Zend_Filter_Input, rather than using the object for what it is probably supposed to do.

How my Zend_Db_Table models look like

I am not too happy with the ZF Frameworks Table access solution. Not that I have anything against the Table Gateway pattern, but in most web applications you almost always have to join data some way or another. Even this rather simple application, a blog, needs joins for displaying articles so that category, comment count and tags can be displayed. Therefore I use an instance of Zend_Db_Table_Abstract only for the simplest purposes and pimp it by using lots of public methods and Zend_Db_Select. An example:

class SomeModel extends Zend_Db_Table_Abstract
{
  $_name = "someTable";
  $_primary = "id";

  public function getSomethingByJoin($param1, $param2)
  {
     $db = $this->getAdapter();
     $select = $db->select();
     ...[build select]
     $result = $db->query($select);

     return $result;
  }
}

That way you still follow the MVC pattern and don’t have to take one compromise after another for getting the Table Gateway to reproduce a result that looks like a join.

PHP Conference Recap: NetBeans, ezComponents, MapReduce..

This years october edition of IPC in Mainz, Germany was the first php conference I have been to. It ended some hours ago and i am quite happy to have been there. I had several great discussions which pointed me to new stuff (to follow) and also met great people.

First thing that completely fascinated me is NetBeans 6.5 with PHP support. It feels much more easy to use than Eclipse PDT. Its faster (from the little tests i could make) and does not strike when you add several large libraries into the include path. It offers much more help on refactoring and organizing than Eclipse PDT does, for example it offers to generate getter/setter, constructor, inheritance methods. Much thanks to Petr who explained to me in detail how NetBeans works.

Ez had lots of people at the conference (to Kore and Tobias i spoke in more detail) and I had some talks with them about ezComponents. This was especially interesting after the great session on object persistence in ez, as well when I heard that ezcMvc, a model-view-controller component, wil be included in the next release of ezc in December 2008. ezComponents has a Database layer, that does not seem to enforce too much overhead on you but still offers great extensibility through a persistence layer and a library that offers to build Database Diffs. The latter feature is really great and I will definitely have a look at this.

Sebastian Bergmann had a great talk on MapReduce which is an old LISP programming paradigm to work with huge datasets. This style was recently revived by Google because tasks can easily be split and distributed for computing on different nodes with hundreds of threads. Implementing a working example in PHP is really easy, just finding a good way to process your large data seems to be a topic that needs lots of thinking.

Additional great things:

• Meet Thomas, the guy that wrote Weaverslave (I use it for years to build simple php applications).
• He is working on a cool open source project Carica Cachegrind with Bastian Feder that will process xdebug cachegrind files via a webinterface (They told me webgrind is actually calculating the numbers wrong).
• Jan Lehnhart and Kore Nordmann had some great things to say about CouchDB.
• Brian Acker talked about Drizzle, which is a forked MySQL that throws out lots of legacy stuff from MySQL out, to build a micro-kernel database server that suites the web.
• Ulf Wendel talked about Mysqli + Mysqlnd. Mysqlnd will be a new internal driver for PHP to access MySQL through Mysqli and optimizes processing and memory consumption. Additionally Mysqli will be extended to offer asynchronous queries that can be fired at the database and polled latter. To be included in PHP 5.3

Not so good things: There were absolutely to few power lines. People using their notebooks were packed on hotspots where power was available. Please more power plug possibilities next time!

My recent ZF ongoings: JQuery, Action Controller, CouchDb

In the last weeks lots of stuff came up for me regarding the Zend Framework. My jQuery component is finished, i have even committed the documentation into the SVN already. Do read it you have to be either an XML fetishist or compile it using a docbook compiler. Lots of questions about ZF and jQuery come up regularly on the mailing lists so I really look forward for the first ZF 1.7 release candidate which will include the helpers for a broader audience. This component comes at a good point, since a few days ago the jQuery team announced a deal with Microsoft and Nokia. Microsoft will include jQuery as the framework to go into its Webbased ASP.Net applications and Nokia will include the library in their mobile phones.

Additionally i reported an important issue (ZF-4385) that offers a patch to create an interface of the Action Controller and therefore allows everyone to implement their own action controller classes. May it be lightweight implementations or like my current use-case webservice only controllers that route all their actions through soap, xml-rpc or a rest server. I would really appreciate if more people would vote +1 on this issue to get it included in 1.7

In the last weeks i also played around with CouchDb, a document based database. Its the perfect storage medium for blogs, wikis or other knowledge-based applications and comes with a lightweight and easy REST Api for ClientSide access. Within the Zym project i managed to implement a prototype that is based on Jurrien’s prototype for a CouchDb client. Matthew of Zend Fame published a proposal for a Zend Couch component yesterday and we teamed up to implement two prototypes (mine is based on Matthews and integrates lots of stuff of the Zym component).

Ox: Cross-correlation coefficients with lags

Working on the homework for my Monetary Economics class I realized that Ox has different mechanisms to calculate correlation coefficients like Mathlab has.

Our assignment was to replicate the Dynamic Correlations graph in Walshs’ Monetary Theory and Policy for European Data, which shows the cross-correlation of different monetary aggregates (M0,M1,M2) with GDP over lagging periods of -8 to +8 quarters.

What was missing for me to do that assignment in Ox was a function to calculate cross-correlation coefficients for two vectors for a specified lag length:

#include <oxfloat.h>

/**
 * Computes the cross-correlation coefficient for two vector series allowing
 * to optionally specify the number of positive and negative lags the ccc
 * should be calculated for.
 *
 * @author Benjamin Eberlei (kontakt at beberlei dot de)
 * @param vVarY Tx1 vector of variable related to
 * @param vVarX Tx1 vector of variable which is tested in different lags
 * @param lags Integer indicating the number of negative and positive lags.
 * @returns (1+lags*2)*1 vector of correlations from -lags to lags
 **/
ccf(const vVarY, const vVarX, const lags)
{
    // Generate positive and negative lags of given length and fill with NaN, so
    // that rows with not available numbers can be dropped from calculating the CCF later on.
    decl mXLag = lag0(vVarX, range(lags, -lags), M_NAN);

    decl mCorr, sCov;

    // initialize result vector holding one crosscorrelation per lag
    decl vCorrLags = zeros(1+lags*2, 1);

    // sadly there are no matrix operations to ease this computation, loop over all lags
    for(decl i = 0; i < 1+lags*2; i++) {
        mCorr = deleter(vVarY ~ mXLag[][i]); // stick y and current lagged x together and delete NaN rows

        // calculate covariance of both time series
        sCov = 1/(rows(mCorr)-1) * sumc( (mCorr[][0]-meanc(mCorr[][0]))' * (mCorr[][1]-meanc(mCorr[][1])) );

        // calculate correlation coefficient
        vCorrLags[i][0] = sCov / ( sqrt(varc(mCorr[][0])) * sqrt(varc(mCorr[][1])) );
    }

    return vCorrLags;
}

It may prove useful to someone.

jQuery Component approved for extras library

Yesterday both my jQuery Helper proposals (Core Helper, UI Widgets) for the Zend Framework have been accepted for development. I have already checked my working prototypes into the Zend Framework SVN. In my opinion the usage is quite stable already. There is likely to be no argument flip, function renaming or whatsoever. I have renamed the jqLink helper to ajaxLink though, because other libraries are offering the same functionality to make Ajax related calls and this way the learning curve for people using different libraries in different projects may be more easy.

What is still missing? I have to add captureStart()/captureEnd() functions to the Container Widgets Accordion and Tabs, and will create additional pane helpers for both of them. This will likely be finished this weekend and then there is only the documentation missing. I am very confident this component will be finished for the Zend Framework 1.7 release later this year.

Since this component also includes helpers for the jQuery UI Widgets 1.6 release (which is not released yet), a little waiting time before release is still good. I hope this all works out and the jQuery UI 1.6 release is in the Google CDN, when ZF 1.7 comes out.

New Calypso DTL Version 0.2

I have got some hints about bugs and optimized the performance of loops a little bit, so that you can now download version 0.2 of the Calypso DTL template engine. Have fun.

Dependency Injection via Interface in PHP: An example

Personal projects and work currently both force me to think about application design and I came across dependency injection the other day. Its quite a complex Design Pattern and hard to grasp, even with the number of examples you can find in the internet. Basically Dependency Injection means you register components that might be a dependency for other components and when loading new classes via the dependency injection framework, it knows of this relationships and assembles the object in need and all its dependencies and returns them fully initialized. This pattern leads to completely encapsulated objects where any class in the object graph can be exchanged for a new implementation without breaking all its dependencies. It also makes testing complex components and object relationships very easy.

As far as I could find out, three different types of Dependency Injection implementations exist: Via Constructor, via Setter and via Interfaces. There exist implementations of Dependency Injection for Java (Spring) that are configured via XML config files and seem very complex. They are also hard to read in code I presume, since you always have to be aware of the current application configuration. There are also some more lightweight implementations (PicoContainer) that assemble their relationships in the application and work with config methods that act as configuration of the dependencies. Martin Folwer also discusses an implementation via interfaces, which seems most accessible for me.

All PHP clones of any dependency injection framework seem to go the complex way via configuration though. With traits and multiple class inheritance on the horizon, some kind of interface injection seems mighty powerful though. So I implemented a really lightweight implementation of Interface Dependency Injection for PHP. The Container works with static method only, is therefore in the global scope, such that configuration is reduced to a minimum. Generally you have to write interfaces for object injection, for example:

interface InjectDbAdapter
{
    public function injectDbAdapter(Zend_Db_Adapter_Abstract $db);
}

Other potential examples include “InjectLogger”, “InjectSoapClient”, or “InjectAppConfig”. You then have to register a component for usage with this interface:

Whitewashing_Container::registerComponent('InjectDbAdapter', Zend_Db::factory($dbConfig));

Now any class that implements the InjectDbAdapter interface can be instantiated via:

$obj = Whitewashing_Container::load('Class');

and the loader takes care of calling the ‘Class’ implementation of injectDbAdapter with the given Database Adapter. A negative consequence of this approach is that you have to implement the inject methods for all interfaces in your concrete class implementations. With Traits (multiple class inheritance) being a new feature in PHP soon, injection via parent classes seems to become a very powerful approach though, which can handle the concrete implementations.

The Container takes care of all the dependency building, so when testing your components you can register lots of mock objects. You can also exchange dependencies for only a subset of objects very easily. I implemented a method “Whitewashing_Container::registerClassComponent”, which registers a dependency component that is used with higher priority in construction of the given class. You can also specify a third parameter $localInterfaceOverride for the highest priority:

$obj = Whitewashing_Container::load('Class', null, array('InjectDbAdapter' => $newDbAdapter));

Speedwise the usage of the Container reduces class instantiation by about 50%, from 0.5 sec for 10000 classes with setting dependencies to 1 sec on my machine. But naturally only classes with dependencies and lots of them should be using this mechanism and with good application design, this shouldn’t be to many. You can download Container and Example sourcecode, to take a look. Currently this Container is not really creating new dependencies for each new class generation but rather inverts the usage of a registry. It should be an easy task to extend the registering method to decide between new class generation and using the globally registered class instance.

Dependency Injection with PHP: Introducing Sphicy

I have written on dependency injection before christmas and how Guice, googles DI framework, offers a simple solution. I copied the functionality for a PHP clone of Guice and named it Sphicy. You saw an early prototype of it in the blogpost mentioned above.

Sphicy configures object dependencies with modules: You explicitly state which implementation should be bound to which interface. An injector then creates instances of these objects via reflection: All constructor dependencies are resolved by looking at the given type hints and initializes those according to the specified bindings.

Two examples included in the source code of Sphicy are Zend Framework and ezComponents MVC bootstrapping modules. Sadly both frameworks default front controllers are engineered in such a way that useful dependency injection needs some workarounds.

As an example I will now discuss the Sphicy Module for Zend Framework MVC applications. To circumvent the singleton and protected Constructor of Zend_Controller_Front, we have to build a new front controller that wraps around it and requires all the dependencies:

class Sphicy_Controller_Front
{
    protected $front;

    /**
     * Create Front Controller for Zend Framework using explicitly dependencies created by Sphicy.
     *
     * @param Zend_Controller_Request_Abstract      $request
     * @param Zend_Controller_Response_Abstract     $response
     * @param Zend_Controller_Router_Interface      $router
     * @param Zend_Controller_Dispatcher_Interface  $dispatcher
     */
    public function __construct(
        Zend_Controller_Request_Abstract $request,
        Zend_Controller_Response_Abstract $response,
        Zend_Controller_Router_Interface $router,
        Zend_Controller_Dispatcher_Interface $dispatcher=null
    )
    {
        $front = Zend_Controller_Front::getInstance();
        $front->setRequest($request);
        $front->setResponse($response);
        $front->setRouter($router);

        if($dispatcher === null) {
            $dispatcher = $front->getDispatcher();
        }
        $front->setDispatcher($dispatcher);
    }

    public function dispatch()
    {
        $this->front->dispatch();
    }
}

You can see that the Sphicy_Controller_Front class requires dependencies in its constructor that are then forward injected into Zend_Controller_Front. You can now create a module that binds all the required dependencies to concrete implementations, for example a module for a Zend MVC Http Application might look like:

class Sphicy_ZendMvc_ExampleModule implements spModule {
     public function configure(spBinder $binder) {
         // Sphicy_Controller_Front does not extend Zend_Controller_Front, because of Singletonitis
         // It offers the dispatch method to proxy against Zend_Controller_Front::dispatch.
         $binder->bind("Sphicy_Controller_Front")->to("Sphicy_Controller_Front");
         $binder->bind("Zend_Controller_Request_Abstract")->to("Zend_Controller_Request_Http");
         $binder->bind("Zend_Controller_Response_Abstract")->to("Zend_Controller_Response_Http");
         // loads all routes
         $binder->bind("Zend_Controller_Router_Interface")->to("MyApplication_Router");
     }
 }

The class MyApplication_Router might look up all the routing information of the application via a hardcoded configuration mechanism. You may say this is a hard dependency, but actually you can just switch modules or implementations at this position to replace the router with another implementation. You can also see that no implementation for the dispatcher is bound. But this dependency is optional and will be created automatically as can be seen in the Sphicy_Controller_Front class.

The front controllers is now created by calling:

$injector = new spDefaultInjector(new Sphicy_ZendMvc_ExampleModule());
$front = $injector->getInstance("Sphicy_Controller_Front");
$front->dispatch();

What happens in the $injector->getInstance() line? Sphicy looks at Sphicy_Controller_Front‘s constructor and finds that four dependencies are needed: Request, Response, Router and Dispatcher implementations. It looks up the bindings and searches for them, creating Zend_Controller_Request_Http, Zend_Controller_Response_Http and MyApplication_Router objects. A dispatcher implementation is not found, but Sphicy recognizes that null is a valid paramater and injects it. The 3 concrete implementations and one null are instantiated and used to construct a valid Sphicy_Controller_Front instance.

You have now stated the dependencies of the Zend Controller Front explicitly and can switch them instantaneously by switching the bindings of interface to implementations in the configuration module.

Have a look at the Sphicy Website and FAQ to see more examples and information about the possibilites of this dependency injection framework.

Zend + jQuery Enhancements: Ajax Forms, Tooltip and Autofill

Some weeks ago Zend Framework 1.7 was released with the jQuery support I contributed. What is the essential advantage of using jQuery support in ZF? You can develop forms with ajax support either by using View Helpers or directly by the integrated Zend_Form support. The implementation of a DatePicker or AutoComplete functionality becomes as easy as using 2-3 lines of php code.

Currently only support for the jQuery UI library is shipped, but you can easily extend the jQuery support on your own and this blog post will show you how using three very popular jQuery plugins: AjaxForm, Tooltip and AutoFill. This will be a series of installments, beginning with the first one: AjaxForms

AjaxForm allows you to enhance any form of yours to submit the data with ajax to the server, so no additional overhead of loading a new page is necessary. Combining the power of Zend_Form and jQuery’s ajaxForm you can even go so far as differentiating between successful and non-validated form submits. We will build a Form Decorator that integrates the AjaxForm plugin in any of your Zend_Form’s. On submit it will send the form data to the server via ajax and clears the form afterwards. Clients that have javascript disabled will work too, the form is submitted to the server in a standard pre-ajax fashion and processed that way.

First what we need is obviously the new decorator, we will call it “My_JQuery_Form_Decorator_AjaxForm” and it will inherit from Zend_Form_Decorator_Form. What we then realize is, that this is just using a view helper to render, so what we need additionally is a “My_JQuery_View_Helper_AjaxForm” that extends from “Zend_View_Helper_Form”. The code of the view helper will have to look as follows to fulfil our needs:

require_once "Zend/View/Helper/Form.php";class ZendX_JQuery_View_Helper_AjaxForm extends Zend_View_Helper_Form{  /**   * Contains reference to the jQuery view helper   *   * @var ZendX_JQuery_View_Helper_JQuery_Container   */  protected $jquery;  /**   * Set view and enable jQuery Core and UI libraries   *   * @param Zend_View_Interface $view   * @return ZendX_JQuery_View_Helper_Widget   */  public function setView(Zend_View_Interface $view)  {    parent::setView($view);    $this->jquery = $this->view->jQuery();    $this->jquery->enable()           ->uiEnable();    return $this;  }  public function ajaxForm($name, $attribs = null, $content = false, array $options=array())  {    $id = $name;    if(isset($attribs['id'])) {      $id = $attribs['id'];    }    if(!isset($options['clearForm'])) {      $options['clearForm'] = true;    }    if(count($options) > 0) {      require_once "Zend/Json.php";      $jsonOptions = Zend_Json::encode($options);      // Fix Callbacks if present      if(isset($options['beforeSubmit'])) {        $jsonOptions = str_replace('"beforeSubmit":"'.$options['beforeSubmit'].'"', '"beforeSubmit":'.$options['beforeSubmit'], $jsonOptions);      }      if(isset($options['success'])) {        $jsonOptions = str_replace('"success":"'.$options['success'].'"', '"success":'.$options['success'], $jsonOptions);      }    } else {      $jsonOptions = "{}";    }    $this->jquery->addOnLoad(sprintf(      '$("#%s").ajaxForm(%s)', $id, $jsonOptions    ));    return parent::form($name, $attribs, $content);  }}

It takes all the form-tag building of the inherited view helper for granted and just appends the necessary jQuery code to the jQuery onLoadActions stack. They will be outputted to the clients browser when calling <?php $this->jQuery(); ?> in your layout or view script. Make sure that you include the jQuery Form plugin in your code, for example with <?php $view->jQuery()->addJavascriptFile(..); >

Programming the decorator becomes a simple trick now:

require_once "Zend/Form/Decorator/Form.php";class My_JQuery_Form_Decorator_AjaxForm extends Zend_Form_Decorator_Form{  protected $_helper = "ajaxForm";  protected $_jQueryParams = array();  public function getOptions()  {    $options = parent::getOptions();    if(isset($options['jQueryParams'])) {      $this->_jQueryParams = $options['jQueryParams'];      unset($options['jQueryParams']);      unset($this->_options['jQueryParams']);    }    return $options;  }  /**   * Render a form   *   * Replaces $content entirely from currently set element.   *   * @param string $content   * @return string   */  public function render($content)  {    $form  = $this->getElement();    $view  = $form->getView();    if (null === $view) {      return $content;    }    $helper    = $this->getHelper();    $attribs    = $this->getOptions();    $name     = $form->getFullyQualifiedName();    $attribs['id'] = $form->getId();    return $view->$helper($name, $attribs, $content, $this->_jQueryParams);  }}

Now to use either the decorator for your form, or just the view helper to print your form tag with jQuery code you can invoke:

$form->addPrefixPath('My_JQuery_Form_Decorator', 'My/JQuery/Form/Decorator', 'decorator');$form->removeDecorator('Form')->addDecorator(array('AjaxForm', array(  'jQueryParams' => array(),)));$view->addHelperPath("My/JQuery/View/Helper", "My_JQuery_View_Helper");$view->ajaxForm("formId1", $attribs, $content, $options);

Now we finished up the view side of our script. Assuming that we use the Form Decorator instead of the View Helper, we can additionally add some fancy logic and error handling ajax fun to the action controller that is handling the Zend_Form instance.

class IndexController extends Zend_Controller_Action{  public function indexAction()  {    $foo = new MyAjaxTestForm();    try {      if(!$foo->isValid($_POST)) {        throw new Exception("Form is not valid!");      } else {        // do much saving and stuff here        if($this->getRequest()->isXmlHttpRequest()) {          $this->_helper->json(array("success" => "SUCCESSMESSAGEHERE"));        }      }    } catch(Exception $e) {      if($this->getRequest()->isXmlHttpRequest()) {        $jsonErrors = array();        foreach( ( new RecursiveIteratorIterator(new RecursiveArrayIterator($form->getMessages())) ) AS $error) {          $jsonErrors[] = $error;        }        $this->_helper->json->sendJson($jsonErrors);      }    }  }}

This has to be processed by a callback function of the AjaxForm and which may for example look like the following which uses a predefined div box (#formMessages, don’t forget to implement it) to render either the success or the error messages.

$form->addDecorator(array('AjaxForm', array(  'jQueryParams' => array(    'success' => "formCallback1",   ),)));$view->jQuery()->addJavascript('function formCallback1(data) {  if(data.errors) {    $("#formMessages").append("<ul>");    for each(var item in data.errors) {      $("#formMessages").append("<li>"+item+"</li>");    }    $("#formMessages").append("</ul>");  } else {    $("#formMessages").html(data.success);  }}');

This seems very complex, but you could include that javascript code into the AjaxForm decorator and implement an Action Helper to do the action controller side of the stuff. This will be an exercise for a future post.

AutoFill and Tooltip extensions will be topic of the next installments of this series, so be aware of new content soonish.

Dependency Injection the juicy way

I have written on dependency injection before and came up with a solution for PHP via interface injection. Thinking about it twice I didn’t like it very much. Its too much overkill that you have to implement all setter methods again and again.

Still dependency injection is the way to write good, testable and easily exchanged and re-wired object graphs. I tried to do lots of dependency injection via constructor lately and realized that it pollutes my constructors when my object graph is too deep.

An example: When I setup my database connection in the bootstrap file and encapsulate it in my model creation factory object, i have to insert the model factory through the configuration into the dispatcher into the different controllers and views to be accessible in the MVC pattern. The model factory has to walk 3 nodes in the object graph without being used at all in the “higher” steps. This creates very unnecessary dependencies.

I came across Misko Hevery‘s Blog, which rocks. There are also some great Google Tech Talks by him, where he argues in favour of dependency injection and debunks singletons as being evil (he does that on his blog too). From there I learnt about Guice, a dependency injection framework for Java by Google.

What I like about Guice: Its easy to use and its immediately obvious to someone without experience, why it works so good and you don’t have to hand down objects deep down the object graph. I cloned the basic functionality for PHP and an example would work as follows.

We first have to implement a module, which defines which concrete implementation should be injected as a placeholder for which interface.

class ServiceModule implements Module{ public function configure(Binder $b) {  $b->bind("Service")->to("ConcreteService");  $b->bind("Add")->to("ConcreteAdd");  $b->bind("Sub")->to("ConcreteSub"); }}

We can now use this module to instantiate an injector object:

$injector = new Injector( new ServiceModule() );$service = $injector->getInstance("Service");

Given that the constructor of ConcreteService would expect an Add and a Sub object, the Injector would realize this and instantiate the concrete implementations ConcreteAdd and ConcreteSub and inject them into the constructor.

What makes this dependency injection so simple and great to use? You can instantiate an injector everywhere in your code and just have to configure it using the additional module implementation. This way you don’t have to make sure that you pass down the dependency injection container from the bootstrap into all nodes of the application. You can also easily work with many frameworks and still be able to use dependency injection without having to hack the whole core of the framework.

My guice clone does more. It allows to pass down additional non-object arguments into constructors, even for object dependencies. It offers a Provider interface to be able to wrap adapters around your already existing ServiceLocator or Registry objects. But its reflection capabilities have to be extended to docblock comments, so that better dependency detection is possible.

Because using dependency injection with only this little example I will refrain from releasing the source code yet. I have to provide some useful documentation for it to be of use to anyone.

ezComponents View Handler for Zend_Pdf

My previous posting discussed different view handlers based on routing information. One example was the PDF View which was implemented rather hackish through overwriting the createResponseBody() function of ezcMvcView. Derick told me the way to go would be writing my own PDF view handler. Since this is a rather lengthy topic I created this new post that only discusses using Zend_Pdf as a View Handler in the new ezComponents MvcTools.

The code for the View Handler would look like the following:

abstract class myPdfViewHandler implements ezcMvcViewHandler
{
    /**
     * Contains the zone name
     *
     * @var string
     */
    protected $zoneName;

    /**
     * Contains the variables that will be available in the template.
     *
     * @var array(mixed)
     */
    protected $variables = array();

    /**
     * Pdf object to be rendered.
     *
     * @var Zend_Pdf
     */
    protected $pdf;

    /**
     * Creates a new view handler, where $zoneName is the name of the block and
     * $templateLocation the location of a view template.
     *
     * @var string $zoneName
     * @var string $templateLocation
     */
    public function __construct( $zoneName, $templateLocation = null )
    {
        $this->zoneName = $zoneName;
    }

    /**
     * Adds a variable to the template, which can then be used for rendering
     * the view.
     *
     * @param string $name
     * @param mixed $value
     */
    public function send( $name, $value )
    {
        $this->variables[$name] = $value;
    }

    /**
     * Processes the template with the variables added by the send() method.
     * The result of this action should be retrievable through the getResult() method.
     */
    public function process( $last )
    {
        // template method
    }

    /**
     * Returns the value of the property $name.
     *
     * @throws ezcBasePropertyNotFoundException if the property does not exist.
     * @param string $name
     * @ignore
     */
    public function __get( $name )
    {
        return $this->variables[$name];
    }

    /**
     * Returns true if the property $name is set, otherwise false.
     *
     * @param string $name
     * @return bool
     * @ignore
     */
    public function __isset( $name )
    {
        return array_key_exists( $name, $this->variables );
    }

    /**
     * Returns the name of the template, as set in the constructor.
     *
     * @return string
     */
    public function getName()
    {
        return $this->zoneName;
    }

    /**
     * Returns the result of the process() method.
     *
     * @return mixed
     */
    public function getResult()
    {
        if($this->pdf instanceof Zend_Pdf) {
            return $this->pdf->render();
        } else {
            throw new Exception("Could not render PDF.");
        }
    }
}

Now you would implement a concrete PDF view handler by extending myPdfViewHandler.

class myConcretePdfViewHandler extends myPdfViewHandler {
    public function process( $last )
    {
        $pdf = new Zend_Pdf();
        // do concrete PDF drawing stuff here

        // save PDF here, will be rendered in getResult()
        $this->pdf = $pdf;
    }
}

And your ezcMvcView implementation will make of createZones() and look like the following:

class myPdfView extends ezcMvcView {
    function createZones( $layout )
    {
        $zones = array();
        // A decision which concrete Pdf Handler should be used would be decided on here.
        $zones[] = new myConcretePdfViewHandler( 'concreteA' );
        return $zones;
    }
}

There you go!

ezComponents 2008.2 Beta - Mvc separation for win

I have written on the new ezComponents MvcTools component before already. Just yesterday the beta of this component was released with the general beta of the 2008.2 version of ezComponents. Several bugfixes and enhancements were included into the MvcTools which make it a perfect component for any Mvc based application.

I reviewed lots of the code myself and can only say i love the beauty of the code. Its very simple but by default the most powerful mvc solution in the PHP market. People working with unittests will like it very much, since all the parts are perfectly separated from each other allowing to test controllers, views, routers and filters in complete separation.

To show one very simple example howto benefit of the separation of view and controllers. If we need a view in both html and pdf, this should generally make no difference for the controller. We add two routes, one for the pdf one for the html view that execute the same controller and action:

class myRouter extends ezcMvcRouter{ public function createRoutes() {  return array(   new ezcMvcRailsRoute( '/pdf', 'SomeController', 'index'),   new ezcMvcRailsRoute( '/', 'SomeController', index' ),  ); }}class SomeController extends ezcMvcController{ public function doIndex() {  $result = new ezcMvcResult();  $result->variables['items'] = Model::retrieveLotsOfItems();  return $result; }}

Now howto decide between PDF and Html view? We use the createView method of our dispatcher configuration, but we still only need the http response writer, nothing more.

class myMvcConfiguration implements ezcMvcDispatcherConfiguration { [...] function createView( ezcMvcRoutingInformation $routeInfo, ezcMvcRequest $request, ezcMvcResult $result ) {  if(strstr($routeInfo->matchedRoute, "/pdf")) {   return new myHtmlView( $request, $result );  } else {   return new myPdfView( $request, $result );  } } function createResponseWriter( ezcMvcRoutingInformation $routeInfo, ezcMvcRequest $request, ezcMvcResult $result, ezcMvcResponse $response ) {  return new ezcMvcHttpResponseWriter( $response ); } [...]}

Now both myHtmlView and myPdfView can create their ezcMvcResponse objects that fill the response body depending on their type. Please note that overwriting createResponseBody() in myPdfView is a shortcut that circumvents me having to write a new PDF View Handler (which would be the way to go).

class myHtmlView extends ezcMvcView { function createZones( $layout ) {  $zones = array();   $zones[] = new ezcMvcPhpViewHandler( 'content', '../templates/index.phtml' );  $zones[] = new ezcMvcPhpViewHandler( 'page_layout', '../templates/layout.phtml' );  return $zones; }}class myPdfView extends ezcMvcView { function createZones() {  // empty, abstract method that has to be defined. } function createResponseBody() {  // Set PDF Content-Type Response Header  $this->result->content->type = "application/pdf";   $pdf = new Zend_Pdf();  // do pdf stuff  return $pdf->render(); }}

Now all the the logic that is potentially in the controller is completely separated from the view handling that may depend on the routing information not on the controller. And views can be tested separately from the controller result. Testability is very high.

REST and Ajax Aware controllers in ezcMvcTools

There are essentially two major different ways to implement a restful application using a web framework. You either implement a router that routes to different controller actions based on the HTTP method used on a requested resource. This is the fancy way, which sadly is not always so practical because many browsers do not support sending more than GET and POST requests. The other way would be to define suburls of a resource such as /user/1/delete for the resource /user/1 and take GET a a request for deleting and POST for the confirmation of the delete.

ezcMvcTools HTTP Request Parser and routing mechanisms currently offer no real help to decide on this issues, but its easy to extend this missing functionality. What we first add are simple checks of the current http request method. We extend ezcMvcHttpRequestParser which will return a derived ezcMvcRequest object that implements 7 new methods: isPost(), isGet(), isDelete(), isPut(), isXmlHttpRequest(), isMethod() and getMethod(). These methods can now be easily used on the request object to determine which action will be undertaken:

class myMvcRequest extends ezcMvcRequest{  public function isPost()  {    return $this->isMethod("POST");  }  public function isGet()  {    return $this->isMethod("GET");  }  public function isPut()  {    return $this->isMethod("PUT");  }  public function isDelete()  {    return $this->isMethod("DELETE");  }  public function isXmlHttpRequest()  {    if(isset($this->raw['HTTP_X_REQUESTED_WITH'])      && strtolower($this->raw['HTTP_X_REQUESTED_WITH']) == "xmlhttprequest") {      return true;    }    return false;  }    public function getMethod()  {    if(isset($this->raw['REQUEST_METHOD']))      return strtolower($this->raw['REQUEST_METHOD']);    }    return false;  }  public function isMethod($method)  {    if(isset($this->raw['REQUEST_METHOD']) &&       $this->getMethod() == strtolower($method)) {      return true;    }    return false;  }}class myMvcHttpRequestParser extends ezcMvcHttpRequestParser{  /**   * Uses the data from the superglobals.   *   * @return ezcMvcRequest   */  public function createRequest()  {    $this->request = new myMvcRequest;    $this->processStandardHeaders();    $this->processAcceptHeaders();    $this->processUserAgentHeaders();    $this->processFiles();    $this->processAuthVars();    $this->request->raw = &$_SERVER;    return $this->request;  }}

This helps us to implement simple decision mechanisms in a single controller action that takes both POST and GET requests. From the point separation of concerns this is not a good design decision. We need routes that can point to different controller actions based on their request method, so that two requests GET /user/1/delete and POST /user/1/delete lead to different methods, for example userController::doDeleteDialog and userController::doDelete. We will simply extend the ezcMvcRailsRoute to support decision based on http request methods:

class myMvcRestRoute extends ezcMvcRailsRoute{  protected $method;  public function __construct( $method, $pattern, $controllerClassName, $action = null, array $defaultValues = array() )  {    $this->method = $method;    parent::__construct($pattern, $controllerClassName, $action, $defaultValues);  }  public function matches( ezcMvcRequest $request )  {    if(strtolower($this->method) == strtolower($request->raw['REQUEST_METHOD'])) {      return parent::matches($request);    }    return null;  }}

This very simple extension is independent of the advanced request parser given above, so you could use it separately. In the light of our delete user example, you would use the new router in the following way:

class myRouter extends ezcMvcRouter{  public function createRoutes()  {    return array(      new myMvcRestRoute( 'GET', '/users/:id/delete', 'UserController', 'deleteDialog' ),      new myMvcRestRoute( 'POST', '/users/:id/delete', 'UserController', 'delete' ),    );  }}

We have now simple rest route support in our ezcMvcTools application.

Bad news: jQuery UI 1.6 ships without Spinner and AutoComplete

The jQuery UI team announced yesterday on its blog that jQuery UI 1.6 will not be shipped with AutoComplete and Spinner Plugin support. No further delay, for the original august 2008 estimated release, is wanted.

This brings about a problem on ZendX_JQuery. All view helpers and decorators that depend on AutoComplete and Spinner Plugins can only be run with SVN trunk or 1.6 release candidates, which have known bugs and problems.

I am adding a compatibility table to the ZendX jQuery manual today that will make its way to the manual of the 1.7.2 release. This will hopefully help and guide everyone to the correct dependencies.

jQuery Helper - What is to include?

I took some time in the last days to develop the concept for a possible jQuery View Helper that could pass the integration into the ZF Core. Since jQuery is organized somewhat different than Dojo is, its possible implementation for ZF will differ.

The jQuery Helper itself will manage inclusion of jQuery javascript files and the base library, much like the Dojo component works (Using the Google CDN). On top of this simple Helpers should mimic $.get, $.post, $.load and $.getJSON in a simple way, so that with a simple function call you can generate an XmlHttpRequest that updates a specified part of the DOM (via injection). (See: CakePHP Ajax)

Problematic are the next ideas: Additional Helpers will allow to specify the jQuery UI Library components (DatePicker, Sortables, Draggable, Dropable..), jQuery Autocomplete or jQuery Form (AjaxForm and AjaxSubmit). These libraries need additional javascript content to be downloaded and implemented in the Zend Framework project. There currently exists no way to implement them using a CDN. Therefore any documentation must clearly specify which additional content has to be installed and how. The helpers will have to be general enough to support this.

My proposal is currently in the workings and will be on the ZF Wiki in the next couple of hours.

Multidimensional Array via SPL

Per default any implementation of ArrayObject does not allow to use the object as multidimensional array. This snippet here implements the support:

class ArrayMultiObject extends ArrayObject
{
    function __construct($array, $flags = 0, $iterator_class = "ArrayIterator")
    {
        $objects = array();
        foreach($array AS $key => $value) {
            if(is_array($value)) {
                $objects[$key] = new ArrayMultiObject($value, $flags, $iterator_class);
            } else {
                $objects[$key] = $value;
            }
        }

        parent::__construct($objects, $flags, $iterator_class);
    }

    public function offsetSet($name, $value)
    {
        if(is_array($value)) {
            $value = new ArrayMultiObject($value);
        }

        return parent::offsetSet($name, $value);
    }
}

ZF jQuery Helper - First Prototype Ready

I checked in the first implementation of my jQuery Helper into my dev svn. It follows my suggested proposal in the Zend Framework Dev Wiki and implements all the advertised features. I also implemented 2 demos that nicely present the functionality.

Please test and comment!

Zend, Dojo and the Django Template Language

With the Zend Framework nearing its 1.6 release and full Dojo Toolkit support I took some time to look up what Dojo is actually capable of. I found the DojoX Django Template Language extension and remembered my neighbor talking about why Python + Django is so much better than PHP with any other framework. So I digged into the Django template language and found that it is quite awesome.

Variable filtering and evaluation looks almost like Smarty the syntax being {{var.key|filter1|filter2:”arg1”:”arg2”}}. The logical syntax is quite different though, taking an somehow object oriented view on templates you can extend an existing template and override specific parts with your more special implementation. Have a look at the following two snippets:

This is an example of Django template inheritance:

{% block helloworld %}Hello World!{% endblock %}

{% extends "base.html" %}

{%block helloworld %}Hello World for Object Oriented Views!{% endblock %}

What does Django do with this second template when rendering? It realizes it inherits logic from a parent template and substitutes all special blocks for the parent ones. With a little object oriented background you can guess the result looks like this:

This is an example of Django template inheritance:

Hello World for Object Oriented Views!

So what was all the talking about Dojo being able to parse this kind of templates? If you envision a helper component that would function like this: 1.) make an ajax request to $url 2.) retrieve json object from the controller 3.) render json object with $template into container $container. The helper would know that the template is needed in this HTML response and appends it to the Dojo Helper output. A link would be generated performing steps 1 and 2, handing over the json data to the template and render the output. What do you get? Templates that can be used Client and Server side.

For example on rendering the view of your blog you can send all the comments using a specific comment building Django template script. Additionally you can also use the same template to render any new comment to the comment list using via an ajax form submit, returning the (model or form) filtered data via JSON and appending it to the comment list using the DojoX DTL parser. For non-JS browsers you can always use a <noscript> variant to render the templates completely server-side.

This generally being a cool idea since it makes developing applications with AJAX technology very easy I began to port the DTL to the Zend Framework and the whole weekend later I got a working implementation that supports at least the “extends”, “for”, “include”, and “comment” tags. As a next task I will implement the helper for DojoX DTL and will report back on my efforts.

Discussing a jQuery Helper for Zend Framework

Yesterday I had some time to test Matthews Dojo View Helper and the Zend_Dojo_Form component. At a first glance the implementation looks great and can almost instantly generate you a very nice form with all the possible Dijit form extensions that Dojo has to offer.

If one were to refactor the Dojo Component to allow for jQuery elements one bigger problem appears. Dojo has a CDN (Content Distribution Network) for all its components, that is, the Zend Dojo components can load all javascript and css files from a distant server. JQuery can only offer its main library to be loaded from a CDN. All additional components, for example the jQuery DatePicker, have to be installed locally. This significantly reduces the possibility for rapid development of Javascript/Ajax/Form components with jQuery and Zend Framework.

One could offer a complete dependency downloadable archive with all the CSS, Javascript and Images inside, but this would be very complex to maintain. Looking at the future Zend Tool capabilities one possibility would be to offer a download client for all the relevant jQuery plugins, but there would have to be a man-middle-server that maintains the most up to date locations of all the plugins, which also has to be maintained. Does anybody have a better solution to solve this problem? Perhaps the jQuery Team needs to be made aware that they need a CDN for their most stable plugins.

Abstracting from the CDN problem, I implemented a simple jQuery View Helper (mostly copy paste and simple rewrites from Matthews Dojo component) and a HtmlElement Form Helper which constructs a Date-Picker from within the template. This is very easy to use and looks great. In the next days I will add further helpers for the jQuery plugins I use in my day to day work live and hope to present a demo. I might even opting for a jQuery proposal aiming at inclusion in the Zend Extras Library.

Update: I got a reply on the jQuery mailing list stating that a CDN is planned for the jQuery UI library. Sadly this does not include plugins for which one might desperately need a View Helper except for maybe the Date Picker. I will post further comments on this issue in the near future.

Finished first Django Template Language Port to PHP + ZF

Today I finished the first almost complete Django Template Language Clone for PHP implementing the Zend Framework View Interface. My proposal is up and running on the ZF Wiki. The source code can be downloaded from my svn repository.