Whitewashing is the blog of Benjamin Eberlei and covers topics in software development. Benjamin works for Qafoo and you can book him for consulting and trainings.

Follow me on twitter Subscribe to RSS My Github Profile

Symfony All The Things (Web)

My Symfony Hello World post introduced the smallest possible example of a Symfony application. Using this in trainings helps the participants understand of just how few parts a Symfony application contains. Sure, there are lots of classes participating under the hood, but I don’t care about the internals only about the public API.

We use microservice architectures for the bepado and PHP Profiler projects that Qafoo is working on at the monent. For the different components a mix of Symfony Framework, Silex, Symfony Components and our own Rest-Microframework (RMF) are used. This zoo of different solutions sparked a recent discussion with my colleague Manuel about when we would want to use Symfony for a web application.

We quickly agreed on: Always. I can’t speak for Manuel, but these are my reasons for this decision:

  • I always want to use a technology that is based on Symfony HttpKernel, because of the built-in caching, ESI and the Stack-PHP project. I usually don’t need this at the beginning of a project, but at some point the simplicity of extending the Kernel through aggregation is incredible.

    This leaves three solutions: Silex, Symfony Framework and Plain Components.

  • I want a well documented and standardized solution. We are working with a big team on bepado, often rotating team members for just some weeks or months.

    We can count the hours lost for developers when they have to start learning a new stack again. Knowing where to put routes, controllers, templates, configuration et al is important to make time for the real tasks.

    This leaves Symfony Framework and Silex. Everything built with the components is always custom and therefore not documented well enough.

  • I want a stable and extendable solution. Even when you just use Symfony for a very small component you typically need to interface with the outside world: OAuth, REST-API, HTTP Clients, Databases (SQL and NoSQL). There is (always) a bundle for that in Symfony.

    Yes, Silex typically has a copy-cat Provider for their own DIC system, but it is usually missing some configuration option or advanced use-case. In some cases its just missing something as simple as a WebDebug Toolbar integration that the Symfony Bundle has.

    My experience with Silex has been that its always several steps behind Symfony in terms of reusable functionality. One other downside with Silex in my opinion is its missing support for DIC and Route caching. Once your Silex application grows beyond its initial scope it starts to slow down.

  • I want just one solution if its flexible enough.

    It is great to have so many options, but that is also a curse. Lukas points out he is picking between Laravel, Silex or Symfony depending on the application use-case.

    But the web technology stack is already complex enough in my opionion. I rather have my developers learn and use different storage/queue or frontend technologies than have them juggle between three frameworks. If my experience with Symfony in the last 4 years taught me anything: Hands-on exposure with a single framework for that long leads to impressive productivity.

    And Symfony is flexible. The Dependency Injection based approach combined with the very balanced decoupling through bundles allows you to cherry-pick only what you need for every application: APIs, RAD, Large Teams. Everything is possible.

The analysis is obviously biased because of my previous exposure to the framework. The productivity gains are possible with any framework as long as it has a flourishing ecosystem. For anyone else this reasoning can end up to choose Laravel, Silex or Zend Framework 2.

So what is the minimal Symfony distribution that would be a starting point. Extending on the Symfony Hello World post:

  1. composer.json
  2. index.php file
  3. A minimal AppKernel
  4. A minimal config.yml file
  5. routing files
  6. A console script
  7. A minimal application bundle

You can find all the code on Github.

Start with the composer.json:

{
    "require": {
        "symfony/symfony": "@stable",
        "symfony/monolog-bundle": "@stable",
        "vlucas/phpdotenv": "@stable"
    },
    "autoload": {
        "psr-0": { "Acme": "src/" }
    }
}

The index.php:

<?php
// web/index.php

require_once __DIR__ . "/../vendor/autoload.php";
require_once __DIR__ . "/../app/AppKernel.php";

use Symfony\Component\HttpFoundation\Request;

Dotenv::load(__DIR__ . '/../');

$request = Request::createFromGlobals();
$kernel = new AppKernel($_SERVER['SYMFONY_ENV'], (bool)$_SERVER['SYMFONY_DEBUG']);
$response = $kernel->handle($request);
$response->send();
$kernel->terminate($request, $response);

We are using the package vlucas/phpdotenv to add Twelve Factor app compatibility, simplyfing configuration. This allows us to get rid of the different front controller files based on environment. We need a file called .env in our application root containing key-value pairs of environment variables:

# .env
SYMFONY_ENV=dev
SYMFONY_DEBUG=1

Add this file to .gitignore. Your deployment to production needs a mechanism to generate this file with production configuration.

Our minimal AppKernel looks like this:

<?php
// app/AppKernel.php

use Symfony\Component\HttpKernel\Kernel;
use Symfony\Component\Config\Loader\LoaderInterface;

class AppKernel extends Kernel
{
    public function registerBundles()
    {
        $bundles = array(
            new Symfony\Bundle\FrameworkBundle\FrameworkBundle(),
            new Symfony\Bundle\TwigBundle\TwigBundle(),
            new Symfony\Bundle\MonologBundle\MonologBundle(),
            new Acme\HelloBundle\AcmeHelloBundle()
        );

        if (in_array($this->getEnvironment(), array('dev', 'test'))) {
            $bundles[] = new Symfony\Bundle\WebProfilerBundle\WebProfilerBundle();
        }

        return $bundles;
    }

    public function registerContainerConfiguration(LoaderInterface $loader)
    {
        $loader->load(__DIR__ . '/config/config.yml');

        if (in_array($this->getEnvironment(), array('dev', 'test'))) {
            $loader->load(function ($container) {
                $container->loadFromExtension('web_profiler', array(
                    'toolbar' => true,
                ));
            });
        }
    }
}

It points to a configuration file config.yml. We don’t use different configuration files per environment here because we don’t need it. Instead we use the closure loader to enable the web debug toolbar when we are in development environment.

Symfony configuration becomes much simpler if we don’t use the inheritance and load everything from just a single file:

# app/config/config.yml
framework:
    secret: %secret%
    router:
        resource: "%kernel.root_dir%/config/routing_%kernel.environment%.yml"
        strict_requirements: %kernel.debug%
    templating:
        engines: ['twig']
    profiler:
        enabled: %kernel.debug%

monolog:
    handlers:
        main:
            type:         fingers_crossed
            action_level: %monolog_action_level%
            handler:      nested
        nested:
            type:  stream
            path:  "%kernel.logs_dir%/%kernel.environment%.log"
            level: debug

We can set the parameter values for %secret% and %monolog_action_level% by adding new lines to .env file, making use of the excellent external configuration parameter support in Symfony.

# .env
SYMFONY_ENV=dev
SYMFONY_DEBUG=1
SYMFONY__SECRET=abcdefg
SYMFONY__MONOLOG_ACTION_LEVEL=debug

Now add a routing_prod.yml file with a hello world route:

# app/config/routing_prod.yml
hello_world:
    pattern: /hello/{name}
    defaults:
        _controller: "AcmeHelloBundle:Default:hello"

And because our routes are dependent on the environment in config.yml also a routing_dev.yml containing the WebDebug toolbar and profiler routes:

# app/config/routing_dev.yml
_wdt:
    resource: "@WebProfilerBundle/Resources/config/routing/wdt.xml"
    prefix:   /_wdt

_profiler:
    resource: "@WebProfilerBundle/Resources/config/routing/profiler.xml"
    prefix:   /_profiler

_main:
    resource: routing_prod.yml

We now need a bundle AcmeHelloBundle that is referenced in routing.yml and in the AppKernel. When we follow Fabiens best practice about adding services, routes and templates into the app/config and app/Resources/views folders adding a bundle just requires the bundle class:

<?php
// src/Acme/HelloBundle/AcmeHelloBundle.php

namespace Acme\HelloBundle;

use Symfony\Component\HttpKernel\Bundle\Bundle;

class AcmeHelloBundle extends Bundle
{
}

And the controller that renders our Hello World:

<?php
// src/Acme/HelloBundle/Controller/DefaultController.php

namespace Acme\HelloBundle\Controller;

use Symfony\Bundle\FrameworkBundle\Controller\Controller;

class DefaultController extends Controller
{
    public function helloAction($name)
    {
        return $this->render(
            'AcmeHelloBundle:Default:hello.html.twig',
            array('name' => $name)
        );
    }
}

Now we only put a template into app/Resources:

{# app/Resources/AcmeHelloBundle/views/Default/hello.html.twig #}
Hello {{ name }}!

As a last requirement we need a console script to manage our Symfony application. We reuse the vlucas/phpdotenv integration here to load all the required environment variables:

#!/usr/bin/env php
<?php
// app/console

set_time_limit(0);

require_once __DIR__.'/../vendor/autoload.php';
require_once __DIR__.'/AppKernel.php';

use Symfony\Bundle\FrameworkBundle\Console\Application;
use Symfony\Component\Console\Input\ArgvInput;

Dotenv::load(__DIR__ . '/../');

$input = new ArgvInput();
$kernel = new AppKernel($_SERVER['SYMFONY_ENV'], (bool)$_SERVER['SYMFONY_DEBUG']);
$application = new Application($kernel);
$application->run($input);

Voila. The minimal Symfony distribution is done.

Start the php built in webserver to take a look

$ php -S localhost:8080 web/index.php

I personally like this simplicity of that, the only thing that annoys me are the two routing files that I need to conditionally load the web profiler routes and the closure loader for the web_profiler extension. I suppose the nicer approach would be a compiler pass that does all the magic behind the scenes.

From this minimal distribution you can:

  1. Add new services to app/config/config.yml.
  2. Add new routes to app/config/routing_prod.yml.
  3. Add controllers into new bundles and templates into app/Resources.
  4. Add third party bundles or Stack-PHP implementations when you need existing, reusable functionality such as OAuth, Databases etc.
  5. Add configuration variables to .env file instead of using the app/config/parameters.yml approach.

This scales well, because at every point you can move towards abstracting bundles and configuration more using Symfony’s built in functionality. No matter what type of application you build, it is always based on Symfony and the building blocks are always the same.

I suggest to combine this minimal Symfony with the QafooLabsFrameworkExtraBundle that I blogged about two weeks ago. Not only will the Symfony be lightweight also your controllers. You can built anything on top this foundation from simple CRUD, APIs, hexagonal- or CQRS-architextures.

Lightweight Symfony2 Controllers

For quite some time I have been experimenting how to best implement Symfony 2 controllers to avoid depending on the framework. I have discussed many of these insights here in my blog.

There are three reasons for my quest:

Simplicity: Solutions to avoid the dependencies between framework and your model typically introduce layers of abstraction that produce complexity. Service layers, CQRS and various design patterns are useful tools, but developing every application with this kind of abstraction screams over-engineering.

While the Domain-Driven Design slogan is “Tackling complexity in software”, there are many abstractions out there that can better be described as “Causing complexity in software”. I have written some of them myself.

Testability: There is a mantra “Don’t unit-test your controllers” that arose because controllers in most frameworks are just not testable. They have many dependencies on other framework classes and cannot be created in a test environment. This lead many teams to use slow and brittle integration tests instead.

But what if controllers were testable because they don’t depend on the framework anymore. We could avoid testing all the many layers that we have removed for the sake of simplicity and also reduce the number of slow integration tests.

Refactorability: I found that when using service layer or CQRS, there is a tendency to use them for every use-case, because the abstraction is in place. Any use-case that is not written with those patterns is coupled against the framework again. Both development approaches are very different and refactoring from one to the other typically requires a rewrite.

A good solution should allow refactoring from a lightweight controller to a service layer with a small number of extract method and extract class refactorings.

While working on the Qafoo PHP Profiler product I went to work on a solution that allowed for Simplicity, Testability and Refactorability and came up with the NoFrameworkBundle.

The design of the bundle is careful to extend Symfony in a way that is easy for Symfony developers to understand. To achieve this it heavily extends upon the FrameworkExtraBundle that is bundled with Symfony.

The design goals are:

  • Favour Controller as Services to decouple them from the Framework.
  • Replace every functionality of the Symfony Base controller in a way that does not require injecting a service into your controller.
  • Never fetch state from services and inject it into the controller instead.
  • Avoid annotations

The concepts are best explained by showing an example:

<?php

use QafooLabs\MVC\TokenContext;

class TaskController
{
    private $taskRepository;

    public function __construct(TaskRepository $taskRepository)
    {
        $this->taskRepository = $taskRepository;
    }

    public function showAction(TokenContext $context, Task $task)
    {
        if (!$context->isGranted('ROLE_TASK', $task)) {
            throw new AccessDeniedHttpException();
        }

        return array('task' => $task);
    }
}

This example demos the following features:

  • The TokenContext wraps access to the security.context service and is used for checking access permissions and retrieving the current User object. It is passed to the controller with the help of ParamConverter feature.

    TokenContext here is just an interface and for testing you can use a very simple mock implementation to pass an authenticated user to your controller.

  • View parameters are returned from the controller as an array, however without requiring the @Template annotation of the SensioFrameworkExtraBundle.

The next example demontrates the abstraction for form requests that help writing very concise form code:

<?php

use QafooLabs\MVC\TokenContext;
use QafooLabs\MVC\RedirectRouteResponse;
use QafooLabs\MVC\FormRequest;

class TaskController
{
    private $taskRepository;

    public function newAction(FormRequest $formRequest, TokenContext $context)
    {
        $task = new Task($context->getUser());

        if ($formRequest->handle(new TaskType(), $task)) {
            $this->taskRepository->save($task);

            return new RedirectRouteResponse('Task.show', array('id' => $task->getId()));
        }

        return array('form' => $formRequest->createFormView());
    }
}
  • The RedirectRouteResponse is used to redirect to a route without a need for the router service.

  • Usage of the FormRequest object that is a wrapper around FormFactory and Request object. It is passed by using a ParamConverter. The method $formRequest->handle combines binding the request and checking for valid data.

    Again there is a set of mock form request that allow you to simulate valid or invalid form requests for testing.

Writing controllers in this way addresses my requirements Simplicity, Testability and Refactorability. For simple CRUD controllers they only ever need access to a repository service. If one of your controllers grows too big, just refactor out its business logic into services and inject them.

Check out the repository on Github for some more features that we are using the Profiler.

Update 1: Renamed FrameworkContext to TokenContext as done in new 2.0 version of the bundle.

Spotting Feature Envy and Refactoring

I was following Aki’s on the SoCraTes2014 conference last week about Legacy Code and Refactoring. In the session a piece of real-world code was shown that contained one of most common code smells in LegacyCode: Feature Envy.

Martin Fowler describes this smell as “a method that seems more interested in a class other than the one it is in. The most common focus of the envy is the data.”

This code smell causes two very similar problems in a code base:

  • Duplication of rules related to their data throughout the code base.
  • Spreading domain concepts throughout the whole code-base.

A simple example can show this problem with DateTime is wide-spread in many PHP projects. We often see code such as the following computation to create a range of dates for a reporting function:

<?php

function calculateReport(DateTime $start, $days)
{
    if ($days < 0) {
        throw new InvalidArgumentException($days);
    }

    $endDate = clone $start;
    $endDate->modify('+' . $days . ' day');

    // calculation code here
}

This is a really simple example for Feature Envy: The computation and validation of creating an end date some days in the future of a start date can easily be duplicated throughout the whole code base and implements business rules that belong to the class DateTime itself:

<?php

class DateTime
{
    // ... other methods

    public function getDateDaysInFuture($days)
    {
        if ($days < 0) {
            throw new InvalidArgumentException($days);
        }

        $endDate = clone $this;
        $endDate->modify('+' . $days . ' day');

        return $endDate;
    }
}

function calculateReport(DateTime $start, $days)
{
    $endDate = $start->getDateDaysInFuture($days);

    // calculation code here.
}

(Note: You cant extend the builtin DateTime this way and need to subclass).

This is a great way to improve your code base massively and achieve what Object-Oriented and Domain-Driven Design is about: Modelling concepts of the domain by encapsulating data and behavior.

Learning to spot this kind of Feature Envy is a very good way to incrementally improve your legacy code base towards better models. I find it very interesting that the focus on fixing technical code-smells duplication and feature envy actually leads to a better domain model. As a side effect it massively deemphasizes DDD building blocks like Entity or Value Object, which in my opinion are a big cause for confusion. Testability increases as well, because data objects with behavior are much simpler to setup for testing.

That is why I recommend learning about Feature Envy. For me it is a very simple approach to find improvements.

Symfony2: A Simple Hello World

Reading Simplyfing Django I was reminded of how we organize the Symfony2 Training at Qafoo to start with a very simple “Hello World” example.

The Symfony2 Standard Edition already contains a huge burden of concepts, assumptions and conventions that can be confusing for a beginner.

  • YAML
  • Twig
  • Environments and Dependency Injection Container
  • Various weird and magic syntaxes for bundles, _controller and template names
  • Various files to open and poke around in
  • Composer
  • Maybe even Doctrine

As a trainer it is very hard to explain all this at the same time. It makes a lot of sense to avoid as many of them as possible.

That is why the Standard Edition is not the best way to start teaching Symfony, but one should actually consider a much lighter version of the framework and then gradually add more and more features until you end up with the standard edition.

Fabien discussed the simplification of the Symfony Standard Edition from another angle before, considering the minimum necessary code to use when reproducing bugs in his blog post “Packing a Symfony Full-Stack Application in one file”.

This blog post is about a truely minimal Symfony edition to start learning about the framework. The first thing I like to show is a really simple Hello World controller:

<?php

namespace Acme\TrainingBundle\Controller;

use Symfony\Component\HttpFoundation\Response;

class HelloController
{
    public function helloAction()
    {
        return new Response('Hello World!');
    }
}

What is the necessary framework glue code to support this Hello World? Lets start from the index file web/index.php:

<?php

require_once __DIR__ . "/../vendor/autoload.php";
require_once __DIR__ . "/../app/AppKernel.php";

use Symfony\Component\HttpFoundation\Request;

$request = Request::createFromGlobals();
$kernel = new AppKernel('dev', true);
$kernel->handle($request)->send();

We are using Composer here and the AppKernel class, lets take a peak at them:

{
    "require": {
        "symfony/symfony": "@stable"
    },
    "autoload": {
        "psr-0": { "Acme": "src/" }
    }
}

The composer.json file installs the latest stable Symfony release and adds autoloading for classes starting with Acme in the folder src.

We need to define a Kernel for our application in app/AppKernel.php and the minimal version of this looks like:

<?php

use Symfony\Component\HttpKernel\Kernel;
use Symfony\Component\Config\Loader\LoaderInterface;

class AppKernel extends Kernel
{
    public function registerBundles()
    {
        return array(
            new Symfony\Bundle\FrameworkBundle\FrameworkBundle(),
            new Symfony\Bundle\TwigBundle\TwigBundle(),
        );
    }

    public function registerContainerConfiguration(LoaderInterface $loader)
    {
        $loader->load(function ($container) {
            $container->loadFromExtension('framework', array(
                'secret' => 'some secret here',
                'router' => array(
                    'resource' => '%kernel.root_dir%/config/routing.yml'
                ),
                'templating' => array('engines' => array('twig'))
            ));
        });
    }
}

In trainings we are starting with a single config.yml file but that is technically not necessary by using the ClosureLoader. The routing component has no such closure loader, so its not possible to avoid YAML in the first minutes of Symfony2 exposure and we reference the app/config/routing.yml:

hello:
  pattern: /
  defaults:
    _controller: "Acme\TrainingBundle\Controller\HelloController::helloAction"

Did you know that you can specify controllers in Symfony by using the static callback syntax? It has disadvantages when considering bundle inheritance, but why bother a beginner with this feature and confuse him with the convention of "AcmeTrainingBundle:Hello:hello".

There are still lots of trip wires to stumble upon in this simple piece of routing.yml:

  • Tabs are not allowed in YAML, in every training at least one person gets bitten by that.
  • 2 vs 4 spaces ambiguity.
  • What is this magic _controller key?
  • I carefully avoid the {} inline syntax of YAML here as well as it introduces yet another ambiguity.

Run this application with the build in webserver:

$ php -S localhost:8000 web/index.php

Hello World!

The next step for a beginner is his first twig template to print the Hello World. To avoid another confusing convention at the beginning, the "AcmeTrainingBundle:Hello:hello.html.twig" template syntax, you can make use of the app/Resources/views/ folder that is automatically registered in the Twig Bundle. Just create a template app/Resources/views/hello.html.twig and change the controller:

<?php

namespace Acme\TrainingBundle\Controller;

use Symfony\Component\HttpFoundation\Response;
use Symfony\Bundle\FrameworkBundle\Controller\Controller;

class HelloController extends Controller
{
    public function helloAction()
    {
        return $this->render('hello.html.twig');
    }
}

The total number of files is 5 (6 with template) for this example. Note that we haven’t actually created a bundle for AcmeWorkshopBundle yet, because with the static syntax we don’t need it.

This is a very good starting point to start exploring more features of Symfony and gradually add the required configuration and files for this.

SOAP and PHP in 2014

“The SOAP stack is generally regarded as an embarrassing failure these days.” – Tim Bray

While the quote by Tim Bray is still true to some degree, the toolstack and possiblities behind SOAP are still superior to REST in my opinion. REST still requires alot of manual coding, where RPC with SOAP allows much automation with tools.

These last years REST has gotten all the buzz, everybody seems to be using it and there are lots of talks about REST on conferences. SOAP used to be big for building APIs, but everybody seems to hate it for various reasons. I have used SOAP in several projects over the last 10 years and this blog post is a random collection of information about the state of SOAP in PHP 2014, as a reminder to myself and to others as well.

Why care about SOAP in 2014? For server to server (RPC) communication it still has massive time to market and stability benefits over REST. The REST toolchain is just not well developed enough, it lacks:

  • a standard to describe the input/output formats of endpoints
  • a way to “just do it”
  • ways to automatically generate clients in any language

While solutions exist for these problems in the REST space, they are often not standardized and don’t serve the full stack and different languages.

WSDLs for SOAP however allow you to generate servers and clients from datastructures by the click of a button or execution of a script. I can get two servers communicating over SOAP, exposing all service methods in literally minutes.

Basics

SOAP is a protocol for Remote-Procedure Calls. HTTP is used as mechanism to talk between client and servers by sending POST requests with XML request and response bodies.

The SOAPServer and SOAPClient objects ship with PHP core by default.

You can expose functions, classes or objects as server by registering service callbacks with SOAPServer#addFunction, SOAPServer#setClass or SOAPServer#setObject and then calling the handle() method, which handles request and response generation and sending in one step. You can normally exit the request directly after handle.

The client is even simpler, it overwrites the __call magic method. Any call to the client therefore looks exactly like the same call on the server in your code.

PHPs Non-WSDL mode

One argument against SOAP is the requirement to define WSDL documents for both server and client to allow communication. This is not true for Clients and Servers both written in PHP. You can use the non-WSDL mode to expose an object from the server and use a non-wsdl client to talk to it. The PHP SOAPClient and SOAPServer have a common exchange format that is used in this case and replaces WSDL entirely.

<?php
// server.php
class MyService
{
    public function add($x, $y)
    {
        return $x + $y;
    }
}

$options = array(
    'uri' => 'http://server/namespace',
    'location' => 'http://server/location',
);

$server = new SOAPServer(null, $options);
$server->setObject(new MyService());
$server->handle();

The client is as simple as the following lines of code:

<?php
// client.php
$options = array(
    'uri' => 'http://server/namespace',
    'location' => 'http://server/location',
);
$client = new SOAPClient(null, $options);
echo $client->add(10, 10);

This kind of services work with flawlessly all datatypes except with objects, which get converted to stdClass with public properties on the Client.

If you are developing internal APIs between components in your own system, then using SOAP in non-WSDL mode is a massive time saver. You can expose remote services for Remote procedure calls this way very easily.

The only downside is that you have no documentation what methods exist on the client and ALL public methods of the server object can be called, so make sure they only have those methods you want to be called remotely.

Debugging SOAP

When something goes wrong in either the client or server, it sucks to debug these problems. In general there are several mechanisms to debug SOAP in PHP:

  1. Enable 'trace' => true option on the SOAPClient. This allows you to call the methods $client->__getLastResponse() and $client->__getLastRequest() to take a look at the interaction between server and client.
  2. When failures happen, SOAPClient throws a SOAPFault exception. You can even throw this exception yourself from the SOAPServer code, and the client can then read this failure. However you must know that the $faultcode variable in the constructor of new SOAPFault($faultcode, $faultmsg) is NOT an integer error code like in normal Exceptions. Instead its either a value SERVER or CLIENT, with the component of the interaction that failed.
  3. If you throw non SOAPFault exceptions from the server, then you need to catch them and recast them to SOAPFault, otherwise the client only sees “Internal Server Error” messages.

You can easily solve the SOAPFault problem by decorating your service with an exception handler, and also logging the errors yourself.

<?php

class SoapExceptionHandler
{
    private $exposeExceptionMessages = array(
        'MyProject\DomainException',
    );

    private $service;

    public function __construct($service)
    {
        $this->service = $service;
    }

    public function __call($method, $args)
    {
        try {
            return call_user_func_array(
                array($this->service, $method),
                $args
            );
        } catch (\Exception $e) {
            // log errors here as well!
            if (in_array(get_class($e), $this->exposeExceptionMessages)) {
                throw new SOAPFAult('SERVER', $e->getMessage());
            }

            throw new SOAPFault('SERVER', 'Application Error');
        }
    }
}

$server = new SOAPServer(null, $options);
$server->setObject(new SoapExceptionHandler(new MyService()));
$server->handle();

Generating WSDLs

SOAP uses a service description format called WSDL to describe the input and output of the server and what methods exist. WSDL are formatted with XML and use XMLSchema to describe the input/output messages. The format is very complex, however tools for any languages allow you to autogenerate WSDLs from code.

There are several reasons to introduce WSDLs for your SOAP service:

  • Your SOAP clients will not be written in PHP, which prevents use of the non-WSDL mode.
  • Clients of the service are used and written by other teams or companies.
  • You want to use the WSDL as a validation mechanism for input from clients.

While you should have some understanding of how a WSDL looks like, you should never write it manually. I use Zend Frameworks SOAP Autodiscovery for this. By default it uses the docblocks @param and @return to generate the correct WSDL for a service:

<?php
$autodiscover = new Zend\Soap\AutoDiscover();
$autodiscover->setClass('MyService')
             ->setUri('http://server/namespace') // same as server 'uri'
             ->setLocation('http://server/soap.php') // same as server 'location'
             ->setServiceName('MyService');
$wsdl = $autodiscover->generate();
$wsdl->dump("/path/to/file.wsdl");

You can now place that WSDL file in any public location and then point both SOAPServer and SOAPClient at the file using the first constructor argument:

<?php
$server = new SOAPServer('http://server/path/wsdl', $options);
$client = new SOAPClient('http://server/path/wsdl', $options);

To make the WSDL generation work with objects and object graphs, you have to use objects in your service API that have only public properties. If you dont do it this way, you will need to convert the objects in a seperate step, something to avoid.

Sometimes you want to use other metadata than docblocks. When using tools like Doctrine you already now much better what datatypes an object has. You can write your own ComplexTypeStrategy to generate the metadata for your WSDL files. This is more advanced topic, but can be understood and automated in a reasonable amount of time.

Generating Objects from WSDL

If you implement a client, you want to generate objects for the datastructures of a WSDL file. You can use those objects instead of the stdClass objects which are used by default.

For this task I use the XSD-TO-PHP library. I normally hack around in the code a little to adjust for correct namespace generation and code-style adjustments, but it works quite well by default. Here is an example of a generated class for the DHL Intraship SOAP API:

<?php
namespace DHL\Intraship;

class Person extends ComplexType
{
  /**
   *
   * @var salutation $salutation
   * @access public
   */
  public $salutation;

  /**
   *
   * @var title $title
   * @access public
   */
  public $title;

  /**
   *
   * @var firstname $firstname
   * @access public
   */
  public $firstname;

  /**
   *
   * @var middlename $middlename
   * @access public
   */
  public $middlename;

  /**
   *
   * @var lastname $lastname
   * @access public
   */
  public $lastname;
}

The next thing you can generate is a classmap, that maps every WSDL Type to your newly generated code, in the above example:

<?php

$client = new SOAPClient($wsdl, array(
    'classmap' => array(
        'Person' => 'DHL\Intraship\Person',
        // all the other types
    )
));

SOAP with different Languages

As long as you stay within the PHP world, SOAP is rather easy with both WSDL and non-WSDL modes. Once you want to talk to Java or C# you need solve some more problems.

The first thing to understand is that SOAP can actually talk in 4 different modes. You can use ‘document’ or ‘rpc’ style, ‘literal’ or ‘encoded’ use. This post on the IBM website describes all the different modes in much detail and I recommend everybody having to work with SOAP to read it.

The essence from that article is, that you will always want to use document/literal for your SOAP services, to be compliant with all languages, wrapping each method call and response in its own Message Document.

However using this style is rather complicated in PHP itself, because for every input and output message you need to create a wrapper object (or array) with a specific structure.

You can fix this problem on the Server by using this DocumentLiteralWrapper class in Zend Framework 2. It has no external dependencies, so you can just copy it into your project if you want.

To generate a WSDL for document/literal mode, use the following methods on Zend Autodiscovery:

<?php
$autodiscover = new Zend\Soap\AutoDiscover();
$autodiscover->setBindingStyle(array('style' => 'document'))
             ->setOperationStyle(array('use' => 'literal'));

Then use the wrapper like such:

<?php

$server = new SOAPServer($wsdl, $options);
$server->setObject(
    new \Zend\Soap\Server\DocumentLiteralWrapper(
        new SoapExceptionHandler(
            new MyService()
        )
    )
);
$server->handle();

SOAP Servers generated this way can be converted into a C# SOAP Client with a bunch of button clicks from Visual Studio. It will generate both the Client object and all the data transfer objects for you. Truely amazing.

Testing SOAP Interaction

Because SOAP is very painful about the exact format of messages and rejects invalid messages in the client already when they do not match the WSDL you certainly want to Integration test your clients and servers.

You can do that in PHPUnit by using a client, that wraps a Server directly and doesn’t require a Webserver. Zend Framework 2 already has such an object, named ZendSoapClientLocal. Its usage is simple:

<?php

$server = new SOAPServer($wsdl, $options);
$server->setObject(
    new \Zend\Soap\Server\DocumentLiteralWrapper(
        new SoapExceptionHandler(
            new MyService()
        )
    )
);
$client = new \Zend\Soap\Client\Local($server, $wsdl);
$client->add(10, 10);

This will pass through the complete SOAP marshalling and unmarshalling process and allow you test SOAP interaction.

If you want to take a look at the code of the Local client, its very easy to achieve this.

Versioning with SOAP/WSDL

If you want to version your SOAP Service, you will need to provide versioned WSDL files on different URLs. You should never change the WSDL at a location, because languages like C# statically create clients from the WSDL, never talking to the WSDL again.

If you take care of your Service objects, then you can design them in a way that you can use the same PHP service object for many different versions of the WSDL file in a backwards compatible way. If your API changes alot, you might need to implement different PHP service classes to allow for versioned APIs.

Conclusion

While the full extent of SOAP and WSDL can be scary, they allow you to write servers and clients for RPC communication between servers and languages very easily. If you don’t need to expose your API to the webbrowser via REST/JSON, then using SOAP is a very good alternative to most of the handcrafting that is necessary for REST APIs.

Assert v2.0: Fluent API and Lazy Assertions

Almost two years ago I started developing a small library Assert (blog post) which contained a set of assertion methods to use in production code. With 18.000 installations from Packagist this is my most successful piece of open source software outside of the Doctrine universe. There is a fair number of contributors and I know several companies using the library in production.

The API however didn’t make me too happy, using the static method calls made it impossible to collect multiple errors and also made the code more verbose than necessary when validating the same value with multiple assertions.

Several weeks ago I stumbled across the Java library AssertJ that gave me the idea how to fix these problems. The last two days I had some time to implement those new functionalities and I am releasing a new version 2.0 of Assert today.

Fluent API

Instead of having to use the static assertion methods, there is now a new fluent API, invoked by calling the function Assert\that($value) and then all the assertions you want to call on that value.

Here are some examples:

<?php

\Assert\that(10)->notBlank()->integer()->range(0, 100);
\Assert\that(array('foo', 'bar'))->isArray()->all()->string();
\Assert\that(null)->nullOr()->boolean();

This new API allows for much shorter and compact assertions.

Lazy Assertions

Using Assert with webforms was never possible unless you wanted to show the user only exactly one error message. Because every assertion fails with an Exception, there was no way to execute multiple assertions and collect the errors. This has changed with the new lazy assertion API, that is similar to the Fluent API:

<?php
\Assert\lazy()
    ->that(10, 'foo')->string()
    ->that(null, 'bar')->notEmpty()
    ->that('string', 'baz')->isArray()
    ->verifyNow();

The method that($value, $propertyPath) requires a property path (name), so that you know how to differentiate the errors afterwards.

On failure verifyNow() will throw an exception Assert\\LazyAssertionException (this does not extend AssertionFailedException) with a combined message:

The following 3 assertions failed:
1) foo: Value "10" expected to be string, type integer given.
2) bar: Value "<NULL>" is empty, but non empty value was expected.
3) baz: Value "string" is not an array.

You can also call the method getErrorExceptions() to retrieve all the underyling AssertionFailedException objects and convert them something useable for the frontend.

Error Messages, Values and Constraints

In version 1.0 Assert did not have default error messages when failures occured. This has changed and now every assertion has a default failure message, as well as access to the value and constraints of an exception:

<?php

use Assert\AssertionFailedException;

try {
    \Assert\that(10)->range(100, 1000);
} catch (AssertionFailedException $e) {
    $e->getMessage(); // Value "10" is not between 100 and 1000.
    $e->getValue(); // 10
    $e->getConstraints(); // array('min' => 100, 'max' => 1000)
}

This helps during development when the assertion exceptions occur and also allows to provide error messages to the user, by using the exception code and the value and constraint data for translating into human readable messages.

You can find more information in the README of Assert.

2013 and 2014

This year was amazing with my new job, various talks on conferences and also for this blog. A total of 15 new posts attracted more than 30.000 unique visitors. Most amazing was all the positive feedback that I got at SymfonyCon in Warsaw with regard to my posts and the projects I have been working on in 2013. Thank you all very much for reading and discussing with me. It has been a very insightful year for me!

The three most viewed posts have been on Symfony with Vagrant (5470), Traits (4450) and Doctrine Repositories (3850).

I am always interested in end of the year reading lists to give me an idea what I might want to read the next year. This years reading list of Matthias Meyer of TravisCI provides many ideas for my next years reading list. My own sad list contains only one technical book that I have read this year, but I think you will appreciate the hint if you havent seen this before: Implementing Domain-Driven Design by Vaughn Vernon. That book has influenced many of my blog posts, discussions and experiments this year and you will see effects of it in 2014 as well.

For 2014 I want to keep up the pace of blogging and complement all those ideas with a sample Symfony application that I want to write. The idea for this project spawned from discussions with readers at various events, mostly at the SymfonyCon in Warsaw. My initial plans were to write an ebook on this issue, but various attempts throughout 2013 ended with motivation failures. This is one reason why I have explicitly put the item “Do NOT write a book” in my 2014 todo list.

Most of my blog posts on architecture are very general and even with the practical Symfony and Doctrine posts it is complicated to see the benefits without a good open source sample. That is why I want to write one and discuss the development on this blog.

To support this effort I would be very grateful if you consider supporting me with a regular Gittip donation. Gittip is a fairly new service that allows you to make very small microdonations to others on a weekly basis.

There is also a roadmap of ideas in my mind for 2014 blog posts, specifically about:

  • Doctrine and Domain-Driven Design
  • Hexagonal Architecture
  • Domain Events

I hope to see you again in 2014 and am very eager for the learnings and discussions that the new year will bring.

Feature Flags and Doctrine Entities

I was in a discussion with some of the EasyBib developers this week about how to use feature flags with Doctrine Entities, Metadata Mapping and their database.

The problem of feature flags with Doctrine is easily explained: If you add properties for a new feature that is disabled in the Doctrine metadata, then you need to upgrade the database before deployment, even when the feature is not being rolled out for some days/weeks. Doctrine requires the database to look exactly like the metadata specifies it.

You can get around this problem by using the loadClassMetadata event quite easily:

<?php
namespace MyProject\Listener;

use MyProject\Config\FeatureFlagConfiguration;
use Doctrine\ORM\Event\LoadClassMetadataEventArgs;

class FeatureFlagMetadataListener
{
    private $featureFlagConfiguration;

    public function __construct(FeatureFlagConfiguration $config)
    {
        $this->featureFlagConfiguration = $config;
    }

    public function loadClassMetadata(LoadClassMetadataEventArgs $eventArgs)
    {
        $classMetadata = $eventArgs->getClassMetadata();

        if ($this->featureFlagConfiguration->isEnabled('super_feature2')) {

            if ($classMetadata->name === "MyProject\\Entity\\User") {
                $classMetadata->mapField(array(
                    'fieldName' => 'superFeatureProperty',
                    'type' => 'string'
                ));
            }

            if ($classMetadata->name === "MyProject\\Entity\\Something") {
                // more mapping
            }
        }
    }
}

You can find documentation on the ClassMetadata in the documentation or API.

Then if the feature is enabled fore some time and accepted into the pool of permanent features, you can remove the listener and move the mapping onto the Entities metadata.

Setting up development machines: Ansible edition

A little over a year ago I posted on using Puppet for setting up development machines. This approach prooved useful as during that time I did setup a new machine several times and benefited from the mostly automatic installation with Puppet. Github is thinking the same on a bigger scale. Earlier this year they introduced Boxen which is using Puppet to setup mac-development machines.

The space of deployment- and configuration management tools is exploding these last years and it is not always feasible to evaluate all of them. The benefit of Puppet over all these tools is its huge userbase and online resources. However the steep learning curve can be a huge pain coupled with the fact that most of the modules online are not reusable at all and mostly serve a documentation for “how not to work with Puppet”. On top of that Puppet is only for configuration management and not suitable for deployment of applications.

Introduction to Ansible

I came across a blog post on Ansible by the awesome guys at Qandidate.com which lead me to investigate Ansible. It is essentially a hybrid between Capistrano and Puppet, solving both the deployment and configuration management problems with a focus on simplicity. In fact the learning curve of Ansible is much more flat than that of Puppet and you can read the whole documentation and understand even the advanced concepts in a few hours.

Ansible is used from a developer- or continuous-integration-machine, which executes tasks on hosts from an inventory. You only need SSH servers running and private keys to connect to them to get it working. With the inventory of hosts to operate on, you can chose to execute ad-oc commands using the ansible command or playbooks using the ansible-playbook command. Playbooks are files written in YAML.

Read Sanders post on the Qandidate blog to get an introduction to Ansible.

Development-Machine Setup

Using Ansible to setup a development machine is a bit pointless, given that Ansible excels at remote task execution and distribution. The benefit in development-machine setup over Puppet is the simplicity of configuration compared to Puppet modules + DSL.

For demonstration I have converted the puppet examples from my blog post on Puppet to Ansible. To start we have to create a folder and create a new file inventory in it containing our local machine:

localhost   ansible_connection=local

This is necessary to convince the remote task execution part of Ansible that localhost exists and there is no SSH necessary to get into that machine.

Now I can create playbooks for the different tools I might want to install and setup, such as Vim in a vim.yml:

---
- hosts: localhost
  tasks:
    - name: Install VIM
      apt: pkg=vim state=present
      sudo: yes
    - name: Install Dotfiles
      git: >
        repo=[email protected]/beberlei/vim-dotfiles.git
        dest=/home/${ansible_user_id}/.vim
    - name: Create .vimrc Symlink
      file: >
        src=/home/${ansible_user_id}/.vim/vimrc
        dest=/home/${ansible_user_id}/.vimrc
        state=symlink
    - name: Compile and Install Command-T plugin
      command: >
        rake make
        chdir=/home/${ansible_user_id}/.vim/bundle/Command-T
        creates=/home/${ansible_uesr_id}/.vim/bundle/Command-T/command-t.recipe

You can see here, tasks are a list of commands to execute. They are always executed in order and stop when the first task in the chain fails. Like in puppet you try to make those tasks idempotent through flags such as creates, signaling when a task needs to be or was already executed. I can optionally use sudo to run commands such as installing packages.

To execute the playbook I call:

$> ansible-playbook -K -i inventory vim.yml

With the -i flag I define the pool of hosts to run on, in our case the local machine and the playbook applies the hosts filter to that inventory. The -K parameter prompts me to enter the sudo password for my Ubuntu machine, otherwise the tasks will fail.

Conclusion

Compared to Puppet, the configuration management of Ansible simplifies a lot of the assumptions for example on ordering and reusability of tasks. For a part-time “devop” like me this is great news, as I don’t need to understand a complex tool and can focus on solving problems.

On top of that the deployment features of Ansible with parallel remote execution over SSH make Ansible a powerful tool that I hope to use much more in the future.

Decoupling from Symfony Security and FOSUserBundle

In this blog post I will show how to decouple your core application from the Symfony Security component and User bundles such as the FOSUserBundle.

In my opinion a framework has to be evaluated by how much it allows you to hide it from your actual application. This blog post will add another perspective on how to achieve decoupling from Symfony user and security with a very simple approach. With this puzzle piece and others I wrote about before (Controllers as Service, Param Converters) and other pieces I still need to write about, I would classify Symfony2 as a good framework.

A number of other authors have written blog posts on decoupling Symfony and your model code as well, such as William Durand on project structure , Matthias Verraes on project structure as well and on Decoupling Forms from Entities.

User management breaks the isolation of your business model open, by introducing the UserInterface from the Security component into your code base. In combination with the FOSUserBundle this can cause a dependency on quite a bit of code already because I wouldn’t call the FOS\UserBundle\Model\User object leightweight. However this and other FriendsOfSymfony bundles are very helpful to get annoying features of your application done in a matter of hours.

You can decouple the FOSUserBundle from our own entities and code by introducing a second object that representes the user concept in our model. The UserInterface entities need fields for username, password, enabled and some more. Except the username and email, the business model rarely needs other properties.

Take an ecommerce system as example, where customers can register as users. In this example we could create two entities MyProject\UserBundle\Entity\User and MyProject\ShopBundle\Entity\Customer. Two strategies exist now:

  1. Use the same table for both entities with some shared and other exclusive columns
  2. Use different tables in the database

I haven’t tried the first option yet, so I cannot say much about the feasibility.

For case two, the User entity looks like this:

<?php
// src/MyProject/UserBundle/Entity/User.php;

namespace MyProject\UserBundle\Entity;

use FOS\UserBundle\Model\User as BaseUser;
use Doctrine\ORM\Mapping as ORM;

/**
 * @ORM\Entity
 * @ORM\Table(name="fos_user")
 */
class User extends BaseUser
{
    /**
     * @ORM\Id
     * @ORM\Column(type="integer")
     * @ORM\GeneratedValue(strategy="AUTO")
     */
    protected $id;
}

The Customer entity is a completly new object, not depending on the UserInterface and containing some duplicated properties.

You can either use exactly the same ID or plant the $userId as correlation ID in the Customer object.

<?php
// src/MyProject/ShopBundle/Entity/Customer.php

namespace MyProject\ShopBundle\Entity;

use Doctrine\ORM\Mapping as ORM;

/**
 * @ORM\Entity
 * @ORM\Table(name="customer")
 */
class Customer
{
    /**
     * @ORM\Id
     * @ORM\Column(type="integer")
     * @ORM\GeneratedValue(strategy="NONE")
     */
    protected $id;

    /**
     * @ORM\Column(type="string")
     */
    protected $username;
}

Now you need to extend the application to synchronize all changes from the User entity to the Customer entity. You can achieve this relatively easy by overwriting the ModelManager or when finally supported by FOSUserBundle listening to events. Your own code can exclusively work with Customer objects now, void of any reference to the Symfony Security component or the huge FOSUserBundle dependency.

blog comments powered by Disqus