Category Archives: Symfony

Dynamic configuration of Doctrine and other services in Symfony

In this post, I illustrate how Symfony’s Expression Languange can be used to dynamically configure services at runtime, and also show how to replace the Doctrine bundle’s ConnectionFactory to provide very robust discovery of a database at runtime.

Traditionally, Symfony applications are configured through an environment. You can have different environments for development, staging, production – however many you need. But traditionally, these environments are assumed to be static. Your database server is here, your memcache cluster is there.

If you’ve bought into the 12 factor app mindset, you’ll want to discover those things at runtime through a service like etcd, zookeeper or consul.

The problem is, the Symfony dependency injection container gets compiled at runtime with a read-only configuration. You could fight the framework and dumped the cached container to trigger a recompilation with new parameters. That’s the nuclear option, and thankfully there are better ways.

Use the Symfony Expression Language

Since Symfony 2.4, the Expression Language provides the means to configure services with expressions. It can do a lot more besides that – see the cookbook for examples – but I’ll focus on how it can be used for runtime configuration discovery.

service.yml for a dynamic service

As an example, here’s how we might configure a standard MemcachedSessionHandler
at runtime. The arguments to the session.handler.memcache service are an expression which will call the getMemcache() method in our myapp.dynamic.configuration service at runtime…

services:
    #set up a memcache handler service with an expression...
    session.handler.memcache:
        class: Symfony\Component\HttpFoundation\Session\Storage\Handler\MemcachedSessionHandler
        arguments: ["@=service('myapp.dynamic.configuration').getMemcached()"]

    #this service provides the configuration at runtime
    myapp.dynamic.configuration:
        class: MyApp\MyBundle\Service\DynamicConfigurationService
        arguments: [%discovery_service_endpoint%, %kernel.cache_dir%]

Your DynamicConfigurationService can be configured with whatever it needs, like where to find a discovery service, and perhaps where it can cache that information. All you really need to focus on now is making that getMemcached() as fast as possible!

class DynamicConfigurationService
{
    public function __construct($discoveryUrl, $cacheDir)
    {

    }

    public function getMemcached()
    {       
        $m = new Memcached();
        $m->setOption(Memcached::OPT_DISTRIBUTION, Memcached::DISTRIBUTION_CONSISTENT);

        //discover available servers from cache or discovery service like
        //zookeeper, etcd, consul etc...
        //$m->addServer('10.0.0.1', 11211);

        return $m;
    }
}

In a production environment, you’ll probably want to cache the discovered configuration with a short TTL. It depends how fast your discovery service is and how rapidly you want to respond to changes.

Dynamic Doctrine Connection

Using expressions helps you configure services with ‘discovered’ parameters. Sometimes though, you want to be sure they are still valid and take remedial action if not. A good example is a database connection.

Let’s say you store the location of a database in etcd, and the location of the database changes. If you’re just caching the last-known location for a few minutes, you’ve got to wait for that to time out before your app starts working again. That’s because you’re not doing any checking of the values after you read them.

In the case of a database, you could try making a connection in something like the ` DynamicConfigurationService` example above. But we don’t expect the database to change often – it might happen one-in-a-million requests. Why burden the application with unnecessary checks?

In the case of Doctrine, what you can do is provide your own derivation of the ConnectionFactory from the Doctrine Bundle.

We’ll override the createConnection to obtain our configuration, call the parent, and retry a few times if the parent throws an exception….

class MyDoctrineConnectionFactory 
   extends \Doctrine\Bundle\DoctrineBundle\ConnectionFactory
{
    protected discoverDatabaseParams($params)
    {
        //    discover parameters from cache
        // OR
        //    discover parameters from discovery service
        //    cache them with a short TTL
    }
 
    protected clearCache($params)
    {
        // destroy any cached parameters
    }

    public function createConnection(
        array $params, 
        Configuration $config = null, 
        EventManager $eventManager = null, 
        array $mappingTypes = array())
    {
        //try and create a connection
        $tries = 0;
        while (true) {
            //so we give it a whirl...
            try {
                $realParams=$this->discoverDatabaseParams($params);
                return parent::createConnection($realParams, $config, $eventManager, $mappingTypes);
            } catch (\Exception $e) {
                //forget our cache - it's broken, and let's retry a few times
                $this->clearCache($params);
                $tries++;
                if ($tries > 5) {
                    throw $e;
                } else {
                    sleep(1);
                }
            }
        }
    }
}

To make the Doctrine bundle use our connection factory, we must set the doctrine.dbal.connection_factory.class parameter to point at our class…

parameters:
     doctrine.dbal.connection_factory.class: MyCompany\MyBundle\Service\MyDoctrineConnectionFactory

So we’re not adding much overhead – we pull in our cached configuration, try to connect, and if it fails we’ll flush our cache and try again. You can add a short sleep between retry attempts, depending on what your database failover characteristics are.

Know any other tricks?

If you’ve found this post because you’re solving similar problems, let me know and I’ll add links into this post.

Byzantime – a historical snippet for every minute of the day

oldbookA couple of days ago, a colleague remarked how his wife was able to relate the time in the morning to a year in the life of the Byzantine Empire….

“0811 [pause]. Year of the Bulgarian massacre of the troops of Nicephorus I”

I needed a little project, and so Byzantime was born – for every minute of the day, it will display a historical event by interpreting the time as a year.

http://byzantime.dixo.net

Under the hood

While it’s fun in itself, what I really wanted to play with was Silex. This is a microframework based on Symfony components, aimed at making single-file apps like this concise and testable.

We’ve been using Symfony at Alexander Street Press for over a year, and I’m impressed at how it’s really helped us raise our game when it comes to quality of engineering. But when you need to make something smaller, it feels like overkill.

So, I gave Silex a try. I’m pretty impressed with the results. It’s certainly concise – to give you an idea, the page needs to request some JSON containing events for the current hour. Here’s how the routing for that is handled using Silex:

$app->get('/event/{year}', function (Silex\Application $app, $year) {

    $start=floor($year/100)*100;
    $end=$start+59;

    $events=$app['historian']->getEvents($start, $end);

    return $app->json($events);

})->assert('year', '\d+');

Only 6 lines of code, but a lot is going on here. Firstly, we’re defining the route for our AJAX request defined as /event/yyyy, where yyyy is the year we’re interested in. This parameter is passed to our handler closure..

The next two lines just do a little arithmetic.

Then we reference $app['historian'], which obtains a previously configured service class from the Pimple dependancy injection container provided by $app. Much like a Symfony service, if we never use it, it won’t be created. Having got the service, I obtain an array of events.

We want to return that data as JSON, and Silex provides a hander helper to do just that.

Finally, you’ll see theres a chained call to assert our year parameter is numeric. If it’s not, the closure would not be executed.

Look at that – it look far longer to describe in English!

Conclusion

I recently dusted off the original pastebin.com code to stick up on github. While PHP gets a fairly bad rap, when I compare old code like that to something taking advantage of current technologies like Silex, Composer and Doctrine, it makes me smile. This is a good time to be working with PHP!

Comments on Byzantime are welcome, I think I’ll probably use it as the basis for further experiments…