Tag Archives: APE

Ajax Push Engine 1.0 released

I’m quite unlucky because I’ve downloaded the APE BETA5 just before they (I mean Anthony Catel) released the 1.0. The quite interesting part is that it fixes all the bugs I’ve talked about sooner. So we could say my last post won’t be very useful to anyone.

Still, if you want to compile APE on CentOS x64, you have have to edit the modules/Makefile file and add the “-L/usr/lib64/mysql” command arg. It will give you something like that :

1
2
3
ifeq ($(HAS_MYSQL), yes)
        MYSQL_FLAGS = -L./deps/mysac/ -I./deps/mysac/ -lmysac -L/usr/lib64/mysql -lmysqlclient_r
endif

They also added some new great demos but they are not included in the 1.0 sources. I’ve been closely following their websites for the last two days and they did an awesome job in a very short time. And I guess a lof of people are taking interest in this project because you can see a lot of monkeys on the APE home page.

As soon as possible, I will make some little test with the very interesting javascript server-side code based on SpiderMonkey.

Ajax Push Engine 1.0 Beta 5

I talked quickly about APE in a recent post. I recently downloaded a new version of their program and
I’ve installed it successfully and it works much better. Since the first released verison, it has become quite easy, you just have to launch “./build.sh”. I had some problems with the linker on my CentOS system and the mysql libraries. But I solved it by added “-L/var/lib64/mysql” to some line of the Makefile.

How to make the demos work
They are some tiny bugs that took me quite some time to bypass :

  • To use the controller demo, you need to put the modules/conf/inlinepush.conf file into bin. I guess this will be fixed someday soon but still, that’s how it works right now.
  • To use the move demo, you need to change the scripts/main.ape.js file and add this line :
    1
    
    include("examples/move.js");
  • To use the TCPSocket demo, you need to copy the modules/conf/proxy.conf file into bin.

Note : Each time you modify a script file (within the “scripts” directoy of APE), you have to restart the server.

The very interesting javascript backend
This project is very interesting because everything in the backend is handled by some little javascript files, the move example file is this one :

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
Ape.registerCmd('setpos', true, function(params, infos) {
        if ((!$defined(params.x) || !$defined(params.y)) && (!isFinite(params.x) || !isFinite(params.y))) return 0;
 
        infos.user.setProperty('x', params.x);
        infos.user.setProperty('y', params.y);
 
        // We get the channel
        var chan = Ape.getChannelByPubid(params.pipe);
 
        // And send the position data of the calling user on it
        if (chan) {
                chan.pipe.sendRaw('positions', {'x': params.x, 'y': params.y}, {'from': infos.user.pipe});
        } else {
                return ['109', 'UNKNOWN_PIPE'];
        }
 
        return 1;
});

This is what allows the client to send this kind of request :

1
[{"cmd":"SETPOS","chl":10,"params":{"x":282,"y":38,"pipe":"3dcaa701e8ee3394d65d5f71cd18e428"},"sessid":"b897476d835d12c150fb9fc337c7c93d"}]

If you don’t add the include line, the answer will be something like that :

1
[{"time":"1260123358","raw":"ERR","data":{"chl":10,"code":"003","value":"BAD_CMD"}}]

The “BAD_CMD” reply means that this commands hasn’t be registered by the Ape.registerCmd method. If you come here because that happened to you, just fix it.

For the Controller demo, this is exactly the same. The file is :

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
Ape.registerCmd("inlinepush", false, function(params, infos) {
        // We check the password ("password" parameter of the "inlinepush.conf" file)
        if (params.password == Ape.config("inlinepush.conf", "password")) {
                // If every request parameters have been correctly defined
                if ($defined(params.channel) && $defined(params.data) && $defined(params.raw)) {
                        // We get the channel
                        var chan = Ape.getChannelByName(params.channel);
                        if (!$defined(chan)) return ["401", "UNKNOWN_CHANNEL"];
 
                        // We send the data (as received) back to the channel
                        chan.pipe.sendRaw(params.raw, params.data);
 
                        return {"name":"pushed","data":{"value":"ok"}};
                } else {
                        return 0;
                }
        } else {
                return ["400", "BAD_PASSWORD"];
        }
})

If you get the the BAD_PASSWORD error response and you’re not sure why, you can change the BAD_PASSWORD line by :

1
return ["400", "BAD_PASSWORD, the right one is :"+Ape.config("inlinepush.conf", "password")];

If you get “BAD_PASSWORD, the right one is :”. There’s a good chance you didn’t move the inlinepush.conf into the bin directory.

Sample communications
Here is a sample chat communication

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
Echange CHAT :
--> [{"cmd":"script","params":{"domain":"ape.webingenia.com","scripts":["http://ape.webingenia.com/Source/mootools-core.js","http://ape.webingenia.com/Source/Core/APE.js","http://ape.webingenia.com/Source/Core/Events.js","http://ape.webingenia.com/Source/Core/Core.js","http://ape.webingenia.com/Source/Pipe/Pipe.js","http://ape.webingenia.com/Source/Pipe/PipeProxy.js","http://ape.webingenia.com/Source/Pipe/PipeMulti.js","http://ape.webingenia.com/Source/Pipe/PipeSingle.js","http://ape.webingenia.com/Source/Request/Request.js","http://ape.webingenia.com/Source/Request/Request.Stack.js","http://ape.webingenia.com/Source/Request/Request.CycledStack.js","http://ape.webingenia.com/Source/Transport/Transport.longPolling.js","http://ape.webingenia.com/Source/Transport/Transport.SSE.js","http://ape.webingenia.com/Source/Transport/Transport.XHRStreaming.js","http://ape.webingenia.com/Source/Transport/Transport.JSONP.js","http://ape.webingenia.com/Source/Core/Utility.js","http://ape.webingenia.com/Source/Core/JSON.js","http://ape.webingenia.com/Source/Core/Session.js"]}}]
<-- (useless html response)
 
// We  try some "CONNECT" and then "JOIN" commands with the name "superfc"
--> [{"cmd":"CONNECT","chl":1,"params":{"name":"superfc"}},{"cmd":"JOIN","chl":2,"params":{"channels":"test"}}]
// Error, the nick name is already use
<-- [{"time":"1260119123","raw":"ERR","data":{"code":"005","value":"NICK_USED"}}]
 
// We  try some "CONNECT" and then "JOIN" commands with the name "Florent"
--> [{"cmd":"CONNECT","chl":5,"params":{"name":"Hello"}},{"cmd":"JOIN","chl":6,"params":{"channels":"test"}}]
// This works !
<-- [{"time":"1260119134","raw":"LOGIN","data":{"sessid":"36a60d006d50882d0d2cd6bcfda90cb9"}},
{"time":"1260119134","raw":"IDENT","data":{"user":{"casttype":"uni","pubid":"8b5fec5af11df915d9806a3be4b46c35","properties":{"name":"Hello"}}}},
{"time":"1260119134","raw":"CHANNEL","data":{"users":[{"casttype":"uni","pubid":"8b5fec5af11df915d9806a3be4b46c35","properties":{"name":"Hello"},"level":1},{"casttype":"uni","pubid":"cc03a08f304a75cc133827c6c4b561c8","properties":{"name":"superfc"},"level":1},{"casttype":"uni","pubid":"541e78a0fbfbc08457c800aacccd212d","properties":{"name":"Florent"},"level":1}],"pipe":{"casttype":"multi","pubid":"2b27244bdb42c193d5158e818ec577d0","properties":{"name":"test"}}}}]
 
// We send 
--> [{"cmd":"SESSION","chl":7,"params":{"action":"set","values":{"currentPipe":"2b27244bdb42c193d5158e818ec577d0"}},"sessid":"36a60d006d50882d0d2cd6bcfda90cb9"}]
<-- [{"time":"1260119136","raw":"LEFT","data":{"user":{"casttype":"uni","pubid":"cc03a08f304a75cc133827c6c4b561c8","properties":{"name":"superfc"}},"pipe":{"casttype":"multi","pubid":"2b27244bdb42c193d5158e818ec577d0","properties":{"name":"test"}}}}]
 
// These next requests are long polls with nothing
--> [{"cmd":"CHECK","chl":9,"sessid":"36a60d006d50882d0d2cd6bcfda90cb9"}]
 
// Nothing new
<-- [{"time":"1260119186","raw":"CLOSE","data":{"value":"null"}}]
 
--> [{"cmd":"CHECK","chl":10,"sessid":"36a60d006d50882d0d2cd6bcfda90cb9"}]
<-- [{"time":"1260119211","raw":"CLOSE","data":{"value":"null"}}]
 
--> [{"cmd":"CHECK","chl":11,"sessid":"36a60d006d50882d0d2cd6bcfda90cb9"}]
<-- [{"time":"1260119239","raw":"CLOSE","data":{"value":"null"}}]
 
// We send some text ("hello")
--> [{"cmd":"SEND","chl":19,"params":{"msg":"Hello hello","pipe":"2b27244bdb42c193d5158e818ec577d0"},"sessid":"36a60d006d50882d0d2cd6bcfda90cb9"}]
// And we receive it ("hello")
<-- [{"time":"1260119399","raw":"DATA","data":{"msg":"hello","from":{"casttype":"uni","pubid":"541e78a0fbfbc08457c800aacccd212d","properties":{"name":"Florent"}},"pipe":{"casttype":"multi","pubid":"2b27244bdb42c193d5158e818ec577d0","properties":{"name":"test"}}}}]

If you don’t understand what is long polling, please look at previous post on the subject.

Please not that that the “CLOSE” RAW isn’t some sort of “session closed” message. I’m not sure this it’s what the developer planned but right now it only means that there’s nothing to receive. So if you get that, don’t panic. Everything is working fine.

If you need any help…
Please, don’t hesitate add some question in the comments, I’ll be happy to help you.

Push (Comet) or pull ?

What is it ?
Most of the modern web applications display recently updated data, and to do that they need to get he latest data very frequently. Some of them even include some real-time chat (Gmail Chat, Facebook chat).

How ?
That’s the interesting part.

  • Pull is pretty dumb. You do a request very frequently and you see if anything new appeared. This consumes some bandwidth, some resources (because server has to check if data has actually changed).
  • Push is going back to the source : Once you’ve made the request on the server, it doesn’t reply instantly. It will wait for something before sending anything. So push over HTTP is in fact a pull with a sleeping response. Using push over HTTP is called Comet.

So pushing data isn’t very complex, it just requires a special server to transmit some data (text, html, xml, or image) over an already opened HTTP connection.

Why do we need some special servers ? Because our current servers aren’t build to make people wait. Some geeks made some example of PHP code being able to make some Comet response but they all use looping usleep. If you used it, you would end up totally killing your server with the growing number of clients.
This is the very interesting part of Comet : It doesn’t consumes more resources, it just requires additional server software.

Two types of push
They are two types of push :

  • The multiple limited time push requests or long poll mode : You make a request and each time you get the response the connection is closed and you have to make an other pull. It’s easier to implement but you could end up doing a lot of http requests, which means generating quite some data.
  • The streaming push mode : You open the connection once, and then data comes asynchronously, and the connection is never closed. If it “accidentally” closes, the client reopens it.

How do do some push enabled web applications
You rethink your software so that it uses the push server.

On PHP, you can use The Ajax Push Engine.
On pure .Net environnement, you can use WCF WsDualHttpBinding to serve data.
The CometD is a stable opensource comet server project. JQuery and Dojo can consume it.
You have LightStreamer, a commercial product with a free license. It can be connected with nearly anything (the only unsupported type of server seems to be PHP) or StreamHub which has a very restrictive free license.

Why some people still use Flex ou SilverLight ?
Http push / Comet requires a complex parallel server infrastructure and it only solves the push (server to client realtime data transmission) problem. When data is received, you still have to treat and display it. Managing this is a lot harder with javascript than it is in Flex ou SilverLight.

By the way, on flex, silverlight, flash, java applets you can use your own sockets to transmit data. This is a very good solution, there a good chance it will consume less bandwidth. But you have to remember that not using the HTTP protocol can create some problems as some companies block every other protocols on their internet gateways.

W3C is on it !
W3C is thinking about adding a WebSocket specification. This could be a very good thing as it would standardize this non-standardized part of the web.

Push outside the web
Push isn’t a web specific concept. Any instant messaging service (like MSN, Jabber, Yahoo or ICQ) does support push.

Any system supporting sockets enables you to make push enabled applications. If you consider the mobile OSes, they all support socket, so they all support push. Androïd supports push by socket, J2ME supports push by sockets, iPhone supports push by Apple push system or sockets and it seems BlackBerry supports it by the BlackBerry push service or sockets.