My dear visitors

Hi my dear visitors,

Some of you seem to like (or fall by mistake in) my unfunny, unspiritual and crappy english speaking blog. I’m very glad you have such bad taste. You are more than 2 500 to come each month. You mostly come about some stuffs around the TC65, SVN, C# .Net or Sharepoint.

By the way, you consumed more than 200 GB of bandwidth this month, mostly on downloading the TC65 SDK. I’m really surprised, I guess a lot of people use this chip even if nearly nobody writes about it.

Posted in English. Tags: , . No Comments »

TC65 HTTP POST request

This is just a quick code to show how to do a simple http POST request from a TC65 chip.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
    private void httpCheck() {
        String data = "Parameter1=000009960623185&Parameter2=$GPWPL,4807.038,N,01131.000,E,WPTNME*5C";
        String url = "http://test.webingenia.com/postIdent";
        HttpConnection conn = null;
        InputStream is = null;
        OutputStream os = null;
        try { // Test HTTP connection
 
            // We prepare the POST request
            conn = (HttpConnection) Connector.open( url );
            conn.setRequestMethod( HttpConnection.POST );
            conn.setRequestProperty( "Content-Type", "application/x-www-form-urlencoded" );
            os = conn.openOutputStream();
            os.write( data.getBytes() );
 
            // We launch the request
            System.out.println( "Response code : " + conn.getResponseCode() );
 
            // We display the generated content
            is = conn.openInputStream();
            int ch;
            System.out.println( "Output : " );
            while ( (ch = is.read()) != -1 )
                System.out.print( (char) ch );
 
        } catch ( Exception ex ) {
            System.out.println( "Http : ex : " + ex.getClass() + " : " + ex.getMessage() );
            ex.printStackTrace();
        } // Whatever happens, we close everything
        finally {
            try {
                if ( conn != null )
                    conn.close();
                if ( is != null )
                    is.close();
                if ( os != null )
                    os.close();
            } catch ( Exception ex ) {
                System.out.println( "Http : ex2 : " + ex.getClass() + " : " + ex.getMessage() );
                ex.printStackTrace();
            }
        }
    }
Posted in English. Tags: , . 4 Comments »

Server-Side HTTP Request tester

I made this simple test website for those of you who would want to check if their HTTP request made from a simple equipment like the TC65 are sent correctly.

The idea is simple :

You can contact me if you don’t understand something, if something doesn’t work or if you would like me to add a feature.

Posted in English. Tags: , . 2 Comments »

.Net, Flex and WebORB.Net

I’ve been working on a project where we had to use Flex on a .Net environment and it had to be realtime. I was a little worried at first that it could be tricky to set up. But with WebORB.Net it’s pretty easy. We used the integrated RTMP (Real Time Messaging Protocol) messaging server. It’s almost like using WCF. The most important differences are that, by default, objects are transmitted as Hashtable and calls can’t be synchronous. We can bind the object to right .Net object within the WebOrb management console but we decided to do it ourself using reflection (because we don’t like to depend too much on the management console).

RTMP
The result is pretty impressive as it makes a really powerful real-time management system. And it’s freaking fast. Like WCF, it can handle concurrent calls. It’s in fact easier because everything is asynchronous but still, it does manage that. These calls are received on parallel threads.

The only problem with WebORB.Net is that you don’t have too much examples in the documentation. Our biggest problem was that we didn’t even knew how to detect what was the current calling connection on a method. This is pretty easy but you can’t exactly guess it. This is a dummy sample to show you how it’s done :

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
public class FlexConnectionsServer:ApplicationAdapter {
	private int _connectionIdCounter = 0;
	private Dictionary<IConnection,int> _connectionsToId = new Dictionary<IConnection,int>();
 
	// Returns the CURRENT connection (must be used from a method called by the flex client)
	private IConnection CurrentConnection { get { return ConnectionHub.getConnectionLocal(); } }
	private int CurrentID { get { return _connectionsToId[ CurrentConnection ]; } }
 
	// When the RTMP service is loaded, we start the time dispatcher thread
	public override bool appStart( IScope app ) {
		new Thread( Thread_TimeDispatcher ).Start();
	}
 
	// This method is called when a client connects
	public override bool appConnect( IConnection conn, object[] parms ) {
		lock ( _connectionsToId ) {
			_connectionsToId.Add( conn, _connectionIdCounter++ );
		}
 
		return true;
	}
 
	// This method is called when a client disconnects
	public override void appDisconnect( IConnection conn ) {
		lock ( _connectionsToId ) {
			_connectionsToId.Remove( conn );
		}
	}
 
	// This is a sample method call
	public String GetMyName() {
		return String.Format( "Client{0}", CurrentId );
	}
 
	// This thread dispatches the current time to every client
	private void Thread_TimeDispatcher() {
		while( true ) {
			var t = DateTime.UtcNow;
			SendCommand( "CurrentTime", new Object[] { t } );
			Thread.Sleep( 1000 );
		}
	}
 
	// Call a remote method on each client connection
	private void SendCommand( String command, Object[] params ) {
		lock ( _connectionsToId ) {
			foreach( var kvp in _connectionsToId )
				( kvp.Key as IServiceCapableConnection ).Invoke( command, params );
		}
	}
}

It’s a little bit like using WCF in Single instance mode.

IIS 7.0 Setup
There was something really weird that happened with my IIS 7.0 on my Windows 7 x64. Using the RTMP connection, I frequently had null return values on .Net method calls. We firstly find a quick way to solve it : Recall each time we get a null value. It’s really dirty, but It worked as a very temporary solution.
Then, when debugging on the IIS process, I noticed a lot of NullReferenceException exceptions displayed on the debugger console. They were about the WebORB “LicenseManager” class, and I had as much exceptions as I had null values on the Flex client. I tried a lot of things that didn’t do anything. And then, I changed the “Managemd pipeline mode” of the Application Pool from “Integrated” to “Classic”, and the problem was solved. The really weird stuff here is that WebORB recommends to use the “Integrated” mode.

MidnightCoders support
We asked MidnightCoders to send us a community edition license, and they instantly answered. We got our license within the 48 hours after our first mail (I was the one slowing it down).

Quick note on the project
To give you a better view of the project I’m talking about :
GPRS Equipment <--(M2MP protocol)–> [Generic M2M Server] <--(WCF)--> [Business logic software] <--(WCF)--> [Flex connections server] <--(RTMP)--> Flex
We’ve got a little equipment (TC65 based as you can guess) that connects to a generic server using our own (real-time and bandwidth optimized) protocol. A business logic software connects to this server using a WCF service. And then we have the WebORB.Net / IIS that connects to this business logic software.

WCF helps make very interesting .Net applications because it simplifies all the communication but can also create deadlocks you might not have seen. Take this example : You have a lock on the WCF Server on the clients list object, you make a call to the WCF client and this client makes a call to the WCF Server that also requires a lock. If you were doing all this client/server calls on the same process this wouldn’t be a problem. But as you use WCF client-to-server and server-to-client calls are made on differents threads. You create a deadlock (unless you set the OperationContract attribute to IsOneWay and/or you fix the deadlock source).

If you’re doing real-time applications, I would recommend to define all the frequently used methods with the “IsOneWay=true” option.

06/11/2009 Update : I added a sample server-to-client code because some people were interested by it.

TC65 Module Exchange Suite (MES) problems

Update (2011-06-28):

You should look at the jOBEXFtp program if you don’t want to spend hours fixing a problem with a crappy software.

When you want to locally deploy software on your TC65 chip, you need the Module Exchange Suite (MES). If it doesn’t work, it won’t tell you why, finding out why can be quite tricky.

If you want to find out why the communication with your TC65 failed, the best way is to use the portmon tool and see what is exchanged between the MESServer.exe and the chip.

I collected 4 portmon captures that could help you diagnose your MES communication problems :

  • Here is a successful MES files listing at 19200 bps. You can see that it sends the “AT+CGMM” at different bit rates unless it gets a good response. Then it sets some serial communication options. And then it sends the “AT^SQWE=100″ commands which fails and then establish a successful link using the “AT^SQWE=3″ command.
  • Here is a failed MES file listing. Here, after the “”AT^SQWE=3″” is sent, MES sends a 26 bytes message and receives a 41 bytes message from the chip. Then it should request the file listing. But here, MES doesn’t request anything. Maybe MES doesn’t work well, or maybe the reply from the chip doesn’t satisfy his requirements.
    Anyway, after that it should send a 27 bytes message containing : “x-obex/folder-listing”, and it should get the file listing. But it doesn’t. It reads on the serial port like if it was still waiting for something. And finally, it closes the connection (with “+++”).
  • Here is a totally failed communication. Here it’s because the modem gets stuck on waiting for the modem hardware flow control (“IOCTL_SERIAL_GET_MODEMSTATUS”) when it’s only a 3 wires (TX,RX,GND) communication. So MES never reads or writes anything.
  • Here is communication that failed by not receiving anything. Everything seems ok but no data was received. It’s quite likely that the RX cable is disconnected.

If you have the same kind of problem, you can send me your portmon capture file.

Posted in English. Tags: , , , . 7 Comments »

SMSOTAP v1.2.1

I have just made a little update to the SMSOTAP program so that the little message generation wizzard can add the NETUSER and NETPWD parameters in the OTAP SM sent to the TC65. Thank you John for this little feature request. This pretty useful, without it you could end up adding a new SM and editing each message.

By the way, I have tested this program on Linux with mono and it works fine.

SMSOTAP 1.2

I made some few changes to the SMSOTAP program for the TC65 :

  • I removed the time limit, it’s stable enough to do not force you to update it frequently.
  • It now uses class 1, PID 7d messages instead of class 0, PID 00 compatibility mode (it doesn’t change anything).
  • It generates OTAP SMS only with the parameters you specifiy and use as few SMS as possible. Most of the time, you can use only one OTAP SMS
  • It will prevent you from sending SMS above 140 chars in 8 bits and 160 chars in 7 bits.
  • I fixed a little bug, I replaced “APORNUM:” by “APNORNUM:”

Configuration file from the previous v1.1 version will still work on this one.

[ Download SMSOTAP 1.2 ]

Thanks to Martijn for his comments on the program. You can send me comments if you would like to see new features added or some bugs corrected.

Posted in English. Tags: , , . 6 Comments »

One weird bug on the XT75

You might have faced this error with your X765 chip. The program crashes with this error :

1
2
3
^EXIT 00010000,02d6414253485f4c434c2c70726f6365647572655265636f72645f702d3e6e756d6265724f66526567697374657265645461736b73203c3d204d41585f4e4f5f524547495354455245445f5441534b53
 
^SHUTDOWN

If you convert the hex array to some text, that will give you :

1
#ABSH_LCL,procedureRecord_p->numberOfRegisteredTasks <= MAX_NO_REGISTERED_TASKS

Which might mean something to someone. But the point is, you won’t find any help. Mostly because it’s an uncommon error.

The error comes from the GPRS connection management. I had it when I was using a wrong APN (with the “AT^SJNET” command) to connect to a host. In my program, connection failed with a classic IOException (“Profile not found”) but 1 or 2 minutes later, the chip was ALWAYS crashed (with the “^EXIT” URC). So, the only solution I found to correct this problem was to automatically detect which APN is required.

This might also improve the ease of deployment of your programs. I like putting as much as possible auto-detection / auto-configuration code as possible. It takes a little time to write it but saves a lot of troubles (last minute configuration, human errors, human explanations, etc.).

The APN auto-detection code is like that :

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
public static String AutoDetectApn( ATCommand atc ) {
 
    String[] apnList = {
        "\"gprs\",\"objcobytel.com\",\"\",\"\",\"88.191.11.20\",0", // Bouygues Telecom : "Objet communiquant" / "ObjetCo" / "ObjCo"
        "\"gprs\",\"m2minternet\",\"\",\"\",\"88.191.11.20\",0",
        "\"gprs\",\"internet-entreprise\",\"orange\",\"orange\",\"88.191.11.20\",0",
        "\"gprs\",\"a2bouygtel.com\",\"\",\"\",\"88.191.11.20\",0",
        "\"gprs\",\"b2bouygtel.com\",\"\",\"\",\"88.191.11.20\",0",
        "\"gprs\",\"ebouygtel.com\",\"\",\"\",\"88.191.11.20\",0",
        "\"gprs\",\"movistar.es\",\"movistar\",\"movistar\",\"88.191.11.20\",0",
        "\"gprs\",\"orange\",\"orange\",\"orange\",\"88.191.11.20\",0",
        "\"gprs\",\"orange.fr\",\"orange\",\"orange\",\"88.191.11.20\",0",
        "\"gprs\",\"websfr\",\"\",\"\",\"88.191.11.20\",0"
    };
 
 
   synchronized (atc) {
        System.out.println( "Waiting 30s..." );
        try {
            Thread.sleep( 30000 );
        } catch (InterruptedException ex) {
            ex.printStackTrace();
        }
 
        for (int i = 0; i < apnList.length; ++i) {
            try {
                String apn = apnList[i];
                System.out.println( "Trying apn " + apn + "..." );
                atc.send( "AT^SJNET=" + apn + "\r" );
 
                SocketConnection conn = (SocketConnection) Connector.open( "socket://88.191.11.20:80" );
                conn.close();
                return apn;
            } catch (Exception ex) {
                System.out.println( "Failed : " + ex.getClass() + " : " + ex.getMessage() );
            }
        }
    }
 
 
    // We couldn't find any APN
    return null;
 
}

And it is only launched when the SIM Card has been changed. To detect that, I have some simple code like this :

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
{ // APN auto-detection
 
    String imsi;
 
    synchronized (atc) {
         imsi = Common.getIMSI( atc );
    }
 
    if (settings.confLastImsi.compareTo( imsi ) != 0) {
        System.out.println( "Sim card changed ! Auto-detecting APN..." );
 
        synchronized (atc) {
            String apn = Common.AutoDetectApn( atc );
 
            if (apn != null)
                reg.confAPN = apn;
        }
 
        settings.confLastImsi = imsi;
 
        // Whatever happens, we still need to save the current IMSI
        settings.Save();
    }
}

Reference :
- The forum that helped me solve this problem
- Translating Hex to Text

TC65 : Slow UDP reception

Receiving UDP datagrams on the TC65 is easy but inefficient. You need to create a new thread if you want to receive data asynchronously. There’s no way of knowing if a new datagram has been received or not without hanging on the ::receive( Datagram ); method. This is quite weird considering you can do it in TCP.

The real problem is it’s freaking slow. The Cinterion documentation just tells you “This method blocks until a datagram is received.”. What it doesn’t say is that nearly each time, it blocks for 100 to 700 ms after UDP datagrams have actually been received. And worse, this slowness/sleeping avoids the program from treating data and finally throws a little java.io.IOException exception : “No buffer space available”. I tried to give the udp receiving thread a higher priority than the other threads, I tried to make the TCP reception thread sleep a lot just in case it would lock some kind of network object, I looked on how could this method be implemented but couldn’t find a solution and don’t believe there’s one.

By the way, most of the codes on UDP data reception are lame. They all show the same stupid synchronous code. The real code you will need is something like that :

You can remove the Logger class calls. It enables me to make some build-specific (mostly logging) code. The “if” false conditions are remove by the java compiler (which is actually mostly a pre-compiler). I will (quickly this time) talk about this someday…

This program isn’t TC65 specific, it should work with any MIDP enabled J2ME device.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
package Network.UDPReceiver;
 
import Common.Logger;
import java.io.IOException;
import javax.microedition.io.Datagram;
import javax.microedition.io.DatagramConnection;
 
/**
 * UDP event-like receiving class
 * @author Florent Clairambault
 */
public class UDPReceiver implements Runnable {
 
    private IUDPReceive _receivingClass;
    private DatagramConnection _udpConn;
    private Datagram _udpDatagram;
    private Thread _thread;
    private boolean _loop;
 
    /**
     * Creates and launchs the receiver
     * @param conn The connection
     * @param receivingClass The receiving class of the UDP receive event
     */
    public UDPReceiver(DatagramConnection conn, IUDPReceive receivingClass) {
        _udpConn = conn;
        _receivingClass = receivingClass;
        init();
        start();
    }
 
    /**
     * Prepares the datagram used to receive data
     */
    private void init() {
        if (Logger.E_VERBOSE)
            Logger.Log( 32, "UDPReceiver.init();" );
        try {
            int maxLength = _udpConn.getMaximumLength();
            if (Logger.E_DEBUG) {
                Logger.Log( 39, "maxLength=" + maxLength );
            }
            _udpDatagram = _udpConn.newDatagram( maxLength );
        } catch (IOException ex) {
 
            // This should NEVER happen !
            if (Logger.E_CRITICAL)
                Logger.Log( 37, "UDPReceiver.init", ex );
        }
    }
 
    /**
     * The actual data reception
     * @return the data received
     * @throws java.io.IOException When something fais, it means we have to stop
     */
    public synchronized byte[] receiveUdpFrame() throws IOException {
        if (Logger.E_DEBUG)
            Logger.Log( 46, "UDPReceiver.receiveUdpFrame();" );
 
        _udpConn.receive( _udpDatagram );
        int size = _udpDatagram.getLength();
        if (Logger.E_DEBUG)
            Logger.Log( 49, "size = " + size );
 
        // We copy the data so that it can be used on an other thread
        byte[] data = new byte[size];
        System.arraycopy( _udpDatagram.getData(), 0, data, 0, size );
 
        // These two lines might seem weird but are the more efficient way
        // to prepare next datagram reception.
        _udpDatagram.reset();
        _udpDatagram.setLength( _udpDatagram.getData().length );
 
        if (Logger.E_DEBUG)
            Logger.Log( 63, "UDPReceiver.receiveUdpFrame : ok !" );
 
        return data;
    }
 
    /**
     * Starts the receiving thread
     */
    private void start() {
        _loop = true;
 
        _thread = new Thread( this, "udp" );
        _thread.start();
    }
 
    /**
     * Plan to stop
     * 
     * This method is in fact useless. The thread will stop when the
     * DatagramConnection (_udpConn) will be closed.
     */
    public void stop() {
        _loop = false;
    }
 
    /**
     * The thread method
     */
    public void run() {
        try {
            while (_loop) {
                if (Logger.E_DEBUG)
                    Logger.Log( 70, "UDPReceiver : running..." );
 
                byte[] data = receiveUdpFrame();
 
                // We "throw" an event-like method
                _receivingClass.UdpDataReceived( data );
            }
        } catch (IOException ex) {
            if (Logger.E_DEBUG)
                Logger.Log( 54, "UDPReceiver.run", ex );
 
            // The connection must have been closed, we have to stop !
            _loop = false;
        } catch (Exception ex) {
            if (Logger.E_CRITICAL)
                Logger.Log( 119, "UDPReceiver.run", ex );
        }
    }
}

The “mother” class has implements this IUDPReceiver interface :

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
package Network.UDPReceiver;
 
/**
 * Interface an asynchronous UDP receiving class must implement
 * @author Florent Clairambault
 */
public interface IUDPReceive {
 
    /**
     * Called when a datagram is received on a connection
     * @param data Content of the datagram
     * 
     * We assume that the receiving class already knows the sender of the
     * datagram because it's a connection established with an "UDP server".
     */
    void UdpDataReceived( byte[] data );
}

The “mother” class should open the connection like that :

1
2
_udpConn = (UDPDatagramConnection) Connector.open( "datagram://" + server );
_udpReceiver = new UDPReceiver( _udpConn, this );

where server is an adresse like that “x.x.x.x:y”. It can also be used as a UDP server with “:y”.

And has to implement the interface IUDPReceiver interface.

Note on receiving UDP over GPRS

Most GPRS connections (every one of them unless you subscribe to special options), are “NATed” (using a NAT router). They are behind a private network (10.x.x.x/8, 192.168.x.x/16, 172.16.x.x/12). That prevents the equipments from being directly accessible (by other means than their phone number).

On TCP, you have to establish a connection with a remote host (that you’re quite likely to call a “server”), and then the server can answer on the link. You have a reliable bi-directional stream you can use to transmit anything you’d like. You can keep the connection open running forever, you just have to send data or set the KeepAlive option. NAT routers break TCP connections when nothing has been transmitted on it for 1 hour to 4 days. This has even been considered like a “bug” by some people because the NAT router reach its memory limit or the max number of TCP connections it can translate (from private to public).
On the TC65, it means you can communicate over a TCP connections for months. You just have to make sure that it’s still alive by providing a little ping mechanism in your protocol (you can send pings every 15/30 minutes for instance).

On UDP, it’s roughly the same thing except there’s no connection. In fact, UDP is minimalist (it only has a 8 bytes header). You have no mechanism for connection establishment, you just throw your data (and by chance, it will be received). The router can’t possibly know when the transmission has ended. So when the router supports bi-directional UDP, it just tries to keep track of the last connections established for a limited time. My personal tomato firmware enabled router, keeps track of the connections when you sent data from LAN to WAN in the last 30 seconds, and that goes to 180 seconds when the other side replies.

So, basically what you need to do to set up a bi-directionnal UDP connection is sending some data from the equipment to the server each 25 seconds. My guess is that even empty UDP packets (8 bytes sized packet) should work.

The most simple/secure way to do this (don’t forget that UDP datagrams can be dropped on their way to the server), is to establish a communication with the server in TCP. Then, ask for a unique identifier (two bytes for instance) and then use it each time you send data on UDP.

If you want to send data between two NATed network, it’s also possible. This little RFC explains it very well. If this seems a little bit to abstract, you should try out wireshark. I always use it when I have a doubts about some low-level network transmission (I like to investigate a lot, just to make sure I’m not missing anything).

I didn’t say it sucks

It didn’t say the TC65 sucks. It’s just not as efficient as I thought it could be. The network stack doesn’t seem to have been built for performances. If you send TCP data or receive UDP data too often, there’s a pretty good chance you will face the same problems. If you found a solution to bypass these problems, please tell me how.

But remember, if you only need to send some data each second (or less often), which is the case in most of the M2M applications, you can be sure the chip will suit your needs.

Posted in English. Tags: , , , , . 3 Comments »

Cinterion TC65 development tools work on Windows 7

I’ve installed Windows 7 on my PC. I made an upgrade from vista. Everything was saved except the deploy/debug connection to the Cinterion’s SDK. But a little repair operation with the setup installer made it work. It’s pretty cool because when I switched from XP to Vista, well… I switched back to XP due to that problem.

22/12/09 update :
MES doesn’t work fine on any other system than XP. You should try the JObexFTP tool instead.

I know I’m talking a lot about the TC65 these last days, but I’m finishing a little program (on my spare time) on this chip for someone. This little project is interesting because it requires to be fully optimized to match the required goals of performance. It’s pretty new for me, because a second delay time was ok in my last projects. Here, data must be sent at 5Hz (each 200 ms) and if possible at 20Hz (each 50ms), and that’s why I talked about the TC65′s crappy TCP stack.

Don’t forget you can download the SDK here.
I don’t provide the SDK anymore, I had to remove it.

Posted in English. Tags: , . 4 Comments »