Recently I interviewed Marc Morera. He is an author of DriftPHP - a new asynchronous non-blocking framework on top of Symfony Components and ReactPHP. We discussed a new framework and how it became possible to make Symfony asynchronous and run it on top of ReactPHP.

- Hi everybody and today we’re gonna have an interview with Marc Morera. He is an author of a new framework for asynchronous PHP and he also organizes the first conference about asynchronous PHP which is gonna happen on 26th 27th March in Barcelona. Hi Marc. How are doing?

Hello. Nice to be here with you today. Thank you.

- What is driftPHP? Why do we need just one more framework in PHP ecosystem? You know there is even a joke that every senior developer should write own framework just to understand how things work. So, what’s the point of drift PHP?

Well, the point here is that there is a small mistake taking consideration that ReactPHP is the same language that PHP itself. Of course, both are based in PHP. And ReactPHP is just a library on top of PHP. But when you model your domain choosing ReactPHP or directly PHP itself without any kind of external library support, your classes, your methods are going to be completely different. From this point taking into consideration that DriftPHP is just another framework is wrong. DriftPHP is the first framework on top of ReactPHP. So if you want to model your entire domain if you want to build an application on top of ReactPHP you won’t find any other framework on top of that technology. So, yeah it’s another framework on top of PHP but if you want to use ReactPHP there is no other framework out there.

- And under the hood, it is based on ReactPHP and Symfony components?

Exactly. I take as well Symfony framework bundle which is the bundle perspective of what Symfony is. And I take ReactPHP components as well managing both components mixing them and taking the skeleton architecture of Symfony. If you work with Symfony you will find that DriftPHP is mostly the same. You can define services, routing, auto wiring, you can define whatever you want the same way that you can do in Symfony, but taking the advantages of working on top of ReactPHP.

- How did you come up with this idea? I have read a series of your articles, where you try to make Symfony kernel asynchronous. You tried to put promises there. Then you built an async kernel and now you have a framework, right?

I’ve been using Symfony for the last eight years or nine or seven I don’t even remember how it started. I like to work on top of Symfony components and the Symfony skeleton. I mean the best applications in my life have been built on top of Symfony so I started some months ago working on top of ReactPHP components as well by doing some stuff with PHP-PM. And by using PHP-PM I understood what ReactPHP was. So from that point of view, I decided that I wanted to make my domain as well on top of ReactPHP. So, you have a server that is on top of ReactPHP, you have a domain that is on top of ReactPHP and if you understand a little bit how ReactPHP works how the event loop works, how promises can be used then it’s going to be very easy for you to just think that if you want the whole application work on top of ReactPHP you need all that small layers to be working with promises. That means that Symfony kernel stops in the wrong way. The promises that are generated from the domain layer, for example, making a Redis call… that promises are stopped in the kernel so you can’t take that promise and put it on the server on the event loop in the server and start making non-blocking calls. And at this point, I decided that the first step was thinking about how to change the control of Symfony. Not changing the kernel itself but allowing the handled method inside the kernel and the HTTP kernel to allow only not only returning responses but allowing as well returning promises of responses. I started with that proof of concept that it was just cloning the kernel of Symfony and adding some small refactorings inside the kernel. Then I had to refactor a little bit how the event dispatcher works because you are going to work as well with promises instead of returning values and that’s it. At the entry point, you handle a request and you can expect a promise of response and on the other side the controller inside Symfony application which s no longer necessary and required to return a response anymore. But you can return that promise generated in your domain layer or infrastructure. Now you have three layers that are built on top of ReactPHP promises. It means that you can have a whole ReactPHP application. So, it started as a proof of concept. I worked so hard to make sure that the proof of concept could be not a small component inside Symfony environment but a starting point of working in that direction. I also saw that Symfony community was not much interested in that field. So, I decided that for all that PHP people working on top of Symfony it would be very interesting to have that alternative to jump to ReactPHP framework called DriftPHP.

- It is not only about the code here inside your application? Now you have also to think that your application becomes the server. There is no Nginx no Apache, your application is the server and you should keep it in mind.

Exactly. It can be Nginx can be an Apache. I’m not sure how it should work right now. But what you have here is that one of the components of DriftPHP is a server itself. You don’t need that process manager that manages your several workers because inside this worker all requests are blocking. You don’t need it anymore because your code is not going to be blocking anymore. You can have like thousands of requests at the same time working with one little server and if the code is properly designed all the requests at the same time are going to work properly with the concurrency. With one server you can have tons of thousands of requests at the same time and no one is going to block anyone.

- And what was the idea? What does drift mean? If it is not a secret.

No, it’s not a secret. It means absolutely nothing. You know, if you go to all the projects I’ve built in my life, I need to admit that for naming I’m not that bad. But if you go to find a name, the reason, the rationality behind the names of my life you will find nothing. I mean what means all the naming in software architecture world but nothing. So DriftPHP… yeah I thought something very fast something, you know like making some drifting maybe. It has the origins on need for speed maybe… I don’t know. But that’s not important.

- So, you sat down… how it should be titled? Drift! Oh yeah, let’s write down!

Exactly! Literally!

- So, we can consider it as a non-blocking Symfony, event-driven Symfony on top of ReactPHP. It is also a built-in server. It can be used without Apache on Nginx. And what are the common use cases? Why I should use Drift and I can’t use for example traditional synchronous Symfony?

Of course the idea of using DriftPHP because you can’t really use another framework it’s a bad idea. I think that most applications nowadays are having the servers we have and having the language we have with PHP 7. I think most applications can be done with any kind of framework if you know how to model good architecture. But what this framework tries to do here is to put an alternative in terms of performance. So, for example, if you want to build e-commerce on top of PHP you can take Sylius. You can take natural PHP any kind of framework. You can take Symfony itself to building such small component, like cart or whatever. But you can also take a framework like DriftPHP and of course, all your models are going to be absolutely the same. But instead of returning values you need to start returning promises. And this is what really makes the software non-blocking. It’s sort of waiting for the results. You’re going to return a promise and you will put that promise in the event loop. This is how works other languages for example golang. In another way of goroutines or whatever or… you know for example NodeJs. For example, in e-commerce or a small application with small microservices, it’s so easy to work that way. You can work with PHPUnit as well. But there may be some small components small libraries inside the PHP environment that are going to be a bit difficult for you if you want to work with that way that on top of ReactPHP. When I say difficult I’m saying quite impossible.

- If I want to write something performant in PHP I should consider some let’s call them pitfalls, yes? Some things that will be… that will come with a lot of pain or it will be even impossible to do async. And what are they?

Yes, for example, you cannot call file_get_contents() what seems something you know… obvious. You should take into consideration here is that when you make a file_get_contents() call… from the moment you call that function until the moment you take the result the PHP threat is completely blocking. Because you have to consider file system as the infrastructure layer as well. So, if you have a unique server with thousands of connections there even we can start talking for example about web sockets. You can have dozens of thousands of web sockets connected in that thread. What happens if you make a file_get_contents() call during that small time? The whole code there, whole web sockets, the whole HTTP connections are going to be blocked. That means that if you go to Doctrine, for example, all the calls in MySQL or SQLite or whatever, all that queries are blocking. So you cannot use Doctrine on top of ReactPHP. And that’s not good news but from the other perspective, we can build something like Doctrine on top of ReactPHP. And we are going to have at that point not blocking DBAL or not blocking ORM. And that is good news from that perspective of ReactPHP.

- If I think about DriftPHP as a Symfony on top of ReactPHP… but actually I can’t just get any component I like, pull it into my project and use it. I should consider what’s inside there, right?

Well, you can have some components on top of Symfony that don’t require non-blocking actions like file_get_contents() or whatever. If you have a file_get_contents() call inside a Symfony component, for example for using a Twig template. Such component has a file_get_contents() call. You can manage how to make that file_get_contents() before the server is actually working and starting listening to the web socket. And you can do that. In fact, DriftPHP has a Twig adapter for working with Twig. Under the hood, this adapter makes all that file_get_contents() calls before the request is handled. So if you want to make something blocking you have to make it at the beginning, before the first request arrives. And as soon as the request is handled by the kernel then the component itself is not going to make such blocking operations anymore. Everything is going to be allocated in memory.

- How does it work under the hood? The application starts, this bootstrapping process executes and then it just keeps alive? We have a running kernel and it runs asynchronously. So, we can think about it as PHP-PM which reduces the bootstrapping time and on top of it we also have performant, a non-blocking asynchronous kernel that keeps running and handles all requests, right?

Well, that’s quite different between that. There’s a big difference between PHP-PM and Drift. PHP-PM is just a collection of Symfony workers that handle requests. And between requests, the same kernel is reused once and again.

- So, Drift is not about reducing the bootstrapping time?

You are actually reusing the same kernel across all the requests, but it’s not only about that. Because in PHP-PM the kernel that exists, it’s Symfony a standard one. So what you will have in the PHP-PM is that it is actually blocking. You have to make one request after the other one. And if you have enough requests at the same time long requests at the beginning and short requests later. Short requests are going to be stopped until the first longer requests are completely fulfilled. So this not going to happen anymore with Drift because it’s not only about reusing the same kernel but of making asynchronous and unlocking requests. So you are taking advantage of both things at the same time.

- Let’s imagine that I already have a Symfony application, it is running on the latest version of Symfony. It is running on the latest version of PHP. It has Opcache but I have a leak of performance. I want to improve even more. Will it be hard to migrate from Symfony to Drift? Because under the hood it is actually the same? There we have Symfony and there Symfony.

It’s not going to be difficult. In fact, there is one application that can be considered as the first application on top of DriftPHP. It was actually built on top of ReactPHP before DriftPHP was actually started. That is apisearch.io. It was built on top of Symfony. Basically, it was Symfony. And I started translating it from Symfony PHP, regular PHP to ReactPHP. And it is actually on top of DriftPHP already. So it was hard to make something like that not because I had a strong layer of functional testing. Thus it was easy for me to be sure that the refactoring was already and properly done. But at the same time changing from Symfony to DriftPHP is like changing from Symfony to Laravel even a bit more harder. You will not have to change the logic of your classes but the way it works. The classes are not going to expect to return the result but a promise of the result. So you are going to change maybe some part of the controller, some part of the commands. But, for example, the configuration layer of your services are going to be exactly the same. You will not have to change the routing. You will not have to change the way you sort your classes you organize your classes inside. It’s going to be exactly the same unless you work for example with Doctrine. In that case, that’s not the best moment for changing to DriftPHP. To be honest and for me, it’s important to say that DriftPHP is not the production-ready framework. DriftPHP is two or three months old. I see that the ReactPHP community needs some time to make that growth to start building some external and third-party libraries like, for example, DBAL or the ORM. But for me, DriftPHP is just the beginning of something like that. We already have a MySQL adapter, Postgres adapter, we have Redis adapter, RabbitMQ adapter, we have a Twig adapter. you have a server, we have an HTTP kernel, an adapter for preloading all the things before the framework request is served. We already have a lot of things but they are not enough to make something for production-ready. So we are in that direction and that means that the more time we have to work in that direction the better performance we’re going to have. The better libraries we are going to work with and of course the more people we have the better and the sooner we are going to to be able to work with production-ready framework. For example, for me, it’s not good to have a production-ready framework if you don’t have a library that works with any kind of metrics third-party service, for example, New Relic. If you can’t go with New Relic then a lot of people and companies are going are not ready for DriftPHP. So that’s a direction we have to take if you want a project ready for production.

- You can try but be ready…

Yes, absolutely. I think it’s important to start digging into how ReactPHP works and for me, it’s the most important thing: what are the advantages of working with ReactPHP instead of PHP. And you will find that maybe now it’s not the right moment to work with that on production. But what if in the future the community was large enough to make something really good in terms of performance. It could be a new stage for PHP community. For the whole PHP community. Not only for Symfony users.

- You also should think that when you migrate then you don’t have an application in the request-response cycle. You should monitor it, use supervisor or something else to monitor that your application is currently running that it doesn’t fail.

Well, not the server itself you should take care of. It’s time the request fails, it should fail inside the scope of the request itself. Not just shutting down the server so the server should be always running even if requests are failing. That should be the direction of the design of the server itself. So you should not have to need any supervisor. Basically because if you put the supervisor inside docker, the entry point is not going to be the server itself. What it’s really important it’s going to be the supervisor. So what can happen here is that you have the supervisor running and all workers just stopped. It’s the same problem that you can have with PHP-PM. So, for me the thing that the server itself and the code itself and the thread that is actually handling your requests is the entry point of docker. If you are working with Amazon Web Services, Amazon load balancer is going to take care that you always have your servers running at the same time with a load balancing. In that case because all requests are non-blocking the load balancing can work directly with round-robin instead of marking it’s working if it’s busy or or it’s not busy.

- So you should keep an eye not on the server itself but the container?

Exactly! But the container, the thread number one in the container. So the entry point is the server itself. And when the server breaks the container is going to break what it is important in terms of docker.

- And you mentioned that when you moved your application ApiSearch from Symfony to Drift you had a lot of tests right? Functional tests. If we consider a request-response cycle it is obvious… but how do you test your application which is always running? Does it differ from traditional testing? Or it is the same?

It is the same. But now you test a component which instead of returning you the value itself returns you a promise. There is a library on top of ReactPHP that is actually built by one of the of the people that has already doing ReactPHP from the scratch. That is Christian Lück. There is a library that it’s called “block”. What you can do if you have a promise you can just wait for the result of the promise and that’s it. Then you can assert with the content of the promise itself. So it’s basically the same. But you need to have that layer of waiting for the promise response.

- You just move from values to promises and you assert that that promise resolves with something or the promise rejects and you check the exception.

Exactly, if you wait for a promise and the promise rejects then the exception is going to be thrown. So it’s exactly the same that you write in a PHPUnit.

- Okay, let’s say that I am brave enough to write my new application in DriftPHP. It’s okay for me to throw away Doctrine, to write these low-level queries by myself. Where should I go? How should I know how to use it? How to write it?

Well, if you want to build your application of top of DriftPHP right now I wouldn’t say you’re brave. I would say you didn’t think so much. For me writing DriftPHP applications today for production… I won’t say “no” but if you want to dig into a framework a little bit use driftphp.io. It’s a documentation itself for the project and you can go to github.com/driftphp and you will find a demo app there. That demo was already built on DriftPHP skeleton and you can find a small application that is actually doing absolutely nothing but saving some values in Redis and making some Twig work. You will find that instead of having three four five ten milliseconds returning requests it actually works in less than 300 microseconds in a non-blocking way. You will find that in terms of performance DriftPHP is much faster than any PHP framework nowadays. But if you go with concurrency you will find that the cover of growing of that concurrency stops maybe on 10 milliseconds even if you go with 1,000 concurrent requests at the same time. The 50% of your requests are going to be returned in 10 milliseconds. If you go with that on any other framework like Symfony you will have… if it works in your computer you will have like tons of hundreds of milliseconds or maybe one second per request.

- Oh, that’s amazing! This demo application, is it just a “hello world” blank page? Or is it a sort of CRUD?

No, you can manage some key-value pairs there by using an adapter for Redis. That is actually tested as far as I remember. You will find there some testing stuff. Well, I’m not sure about what I’m saying I’m going to review it later. If it doesn’t exist I’m going to add some testing there to make sure that people understand how to test promises. But you will find that you can really call some endpoints to make sure that you can add some values. Having a key you can request and you can list all values and query the keys at the same time. And you can delete it. Like a small REST application there. You can also use some Twig to show that even if you would use Twig the performance is not decreasing.

- Everything is already loaded into memory?

Yeah, exactly! There is one component on top of DriftPHP that is called preloading. It asks to all other components that actually are loaded inside of PHP and it requests them to preload their services, to preload their classes, to preload everything they have to promote before the first request is handed.

- And having this sort of architecture now on top of Symfony, on top of this asynchronous kernel… what currently is the slowest part here? In this request-response cycle. What is the slowest part? Do you think it can be even more improved?

There is no slow part here. The slowest part here into your application is going to be the peak of it. PHP language is not asynchronous. You will always do one thing before the others. So what you have to think here is that the hard consumption here is going to be about CPU. Of course in memory as well. But you can read the maximum memory and then if you have a good policy of growing and killing servers you’re going to be able to do that. To solve that on top of Amazon Web Services. But what you have to think here is that tons of thousands of requests are going to use the same CPU. One CPU, there is one threat in PHP. So you don’t have that Nginx or Apache or PHP-PM that is actually multi-threading. You will not have anything like that. You will have one CPU just at the same time across all the things that are happening there: your web sockets, whatever. And if you have a long-running code during that time the thread of PHP is going to be blocked by everyone. And you should consider that. The part that can be slow and I think that in the future the thing that is going to be improved in terms of performance is the event loop. Take into consideration that event loop is a class that is actually making ticks… ticks… ticks… ticks… until the end of time. So the faster that ticks are done the better the performance is going to be across all the actions that are done inside the server.

- We are talking here about hard computations, right? When CPU is utilized and how can I fix it? If I have four CPUs I can just run four containers and they will work?

Exactly. You can have four containers inside that server and then having a load balancer that is just balancing between four. The good thing if you have for example four containers you will be able to run the four containers at the same time using 4 CPUs because at the end of the day it’s going to be multi-threading but using 4 threads of PHP. PHP is not multi-threaded but it’s the way that you can’t really use the 100% of resources of your server.

- But now we only talked about request-response cycle… and we talked about HTTP Kernel. Can I write asynchronous long-running console commands in DriftPHP?

Yes, you can do that. For me console things it’s not very important to be asynchronous in terms of request-response in terms of application. I will consider making a console on top of ReactPHP to be sure that all the actions, all the I/O operations inside my infrastructure layer are done at the same time and in the concurrent way. You know things are easier if you go with a console instead of a service. If you go at the console in Symfony it’s actually using the same kernel that is using the server, the web server. We have a talk about how to work with console components on top of ReactPHP at the conference. So I think it’s going to be interesting to see how Michael actually works with a console.

- Okay, that was interesting. I don’t have any questions. I hope that the talk was interesting to others and I think that the main point here is that PHP can be asynchronous. That in community we have tools to write asynchronous code. We have these low-level ReactPHP components. Now we even have a framework where we don’t have to think about event loop, about this low-level stuff.

Exactly. If people need more information about what we are talking about, we have a small list of resources for ReactPHP. Not for DriftPHP but for ReactPHP. For me, it’s the reason that we are already today here. To understand more how ReactPHP works. If you go to the DriftPHP repository on github you will find also this ReactPHP list there. And at some time you can find like different groups of slack and gitter with some small communities that are working on top of DriftPHP and ReactPHP in different channels. So I’m sure that people are going to be interested and should take consideration that in terms of DriftPHP maybe for next year we will have something really good for production. And we are taking that work because we want to make it happen for sure.

- And just one last question. Why ReactPHP? I know that you have chosen Symfony because you have worked with Symfony for a lot of years. But currently in async PHP community we have several tools: we have ReactPHP, we have AMP and we have Swoole which is sort of something different. Why ReactPHP and not others?

I would say the same response that I told you with the name of the project. Why not? I mean I started digging with ReactPHP because PHP-PM and I was very impressed with all the components that are actually built on top of PHP without the need for any PHP extension. And for me, it’s very good because you can start from scratch doing anything… I’m sure that Swoole and Amp are good libraries as well. And that you can take a good performance numbers by using those libraries. Maybe some other guy in another part of the world is going to have the same idea but instead of ReactPHP uses Swoole and if you ask him or her the same question the response is going to be exactly the same. What’s totally fair because I think that we don’t have enough time to start digging across and checking all the possibilities of everything because it’s my life and it is important as well. If I want to take care of something like driftPHP or the conference I don’t need taking those considerations but taking one and working so hard on that direction otherwise there’s no time for anything else to be honest.

- Thank you for your time and for answering questions.

Thank you very much for you and I hope we meet again soon. Bye.

- Yes, we have a lot to talk yeah. Sure. Bye.