AS with Nokia N900 - workaround (partial fix)

[tomster]

Here's my first impression of the N900 and Scalix's AS:
To make it short: My just unboxed N900 won't sync at all.

I get a notice on the phone: "Error on Exchange server".

Not as if that would give me a whole lotta information, but reading through the tomcat logs would bring up:

Code: Select all

FATAL [WirelessServlet.doPost:127] SERVLET - deviceId: IMEI - Uncaught and unhandled exception was reported by the server:
java.lang.NullPointerException
at com.scalix.wireless.controller.entity.SyncRequest.getCollectionType(SyncRequest.java:60)
        at com.scalix.wireless.controller.command.SyncCommand.execute(SyncCommand.java:126)
        at com.scalix.wireless.handlers.WirelessRequestImpl.execute(WirelessRequestImpl.java:176)
        at com.scalix.wireless.HttpRequestHandler.handleRequest(HttpRequestHandler.java:138)
        at com.scalix.wireless.WirelessServlet.doPost(WirelessServlet.java:113)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:647)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
        at org.springframework.web.servlet.mvc.ServletWrappingController.handleRequestInternal(ServletWrappingController.java:158)
        at org.springframework.web.servlet.mvc.AbstractController.handleRequest(AbstractController.java:153)
        at org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter.handle(SimpleControllerHandlerAdapter.java:48)
        at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:875)
        at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:809)
        at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:571)
        at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:511)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:647)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
        at org.springframework.web.filter.AbstractRequestLoggingFilter.doFilterInternal(AbstractRequestLoggingFilter.java:156)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:172)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:174)
        at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:200)
        at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:283)
        at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:773)
        at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:703)
        at org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:895)
        at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689)
        at java.lang.Thread.run(Unknown Source)

This is not excatly what would help me any further, especially because I ran into sort of the same log files with my E71 at the beginning.
Strangely after a couple of tries the E71 would just work flawlessly... Not so the N900.
I did investigate a bit though on the N900 and found out that obviously Maemo's Mail for Exchange is supposed to work with Exchange 2007 only.

The release notes state different EAS versions to be supported in Scalix 1.0.1
2.5 full
12.0 full (I guess that's MS Exchange 2003)
12.1 partially (I guess that's MS Exchange 2007)

Obviuosly the Nokia would want to use exactly the parts of 12.1 not being implemented by Scalix right now. Bummer!

Florian, you would not know what's the roadmap on fully 12.1 integration, would you?

[jaydenrussell]

I believe a new version 11.4.6 will be out mid month or a early xmas present. Same with AS 1.0.2

Supposed to be working with better AS support, which is the majority of the issues most customers have. I am checking ever day so we can have functional smartphones

[tomster]

I hope your believes will come true...

It's sorta sad to see that N900 sitting in its box watching the "old" E71 do the job meanwhile.

Tom - counting the days till mid month

[sla]

I hope your believes will come true...
It's sorta sad to see that N900 sitting in its box watching the "old" E71 do the job meanwhile.
Tom - counting the days till mid month

Also keep in mind that the n900 is the first nokia phone running maemo. There are still quite a
few bugs some of which are supposedly being fixed by a firmware update real soon.

--
sla

[tomster]

Just upgraded Scalix to 11.4.6 and AS to 1.0.2.
No improvement. The log-messages state exactly the same.
Seems like I'd have to wait what update Nokia comes up with...

[tomster]

Nokia released a FW update (PR1.1) for the N900 today. The new FW includes a newer version of MfE (Mail for Exchange) client with now added Exchange 2003 support.
So let me tell you what improved with the newer version:

With PR1.1 the N900 would now finally start initial contact with Scalix. Yippieh!
My first try was a full-sync from scratch- but emails only. No calendar/tasks or contacts.
MfE won't let you choose the folders to be synced so I ended up staring at the phone's
status bar for 15 Minutes to observe each email being pushed onto the phone correctly.
Everything seemed to work OK, the synced finished.

BUT opening the N900 email client (Modest) showed nothing but empty folders.

A second (manually triggered) sync would bring up the following message in scalix-wireless.log:

Code: Select all

FATAL [WirelessServlet.doPost:127] SERVLET - deviceId: IMEI - Uncaught and unhandled exception was reported by the server:
com.scalix.wireless.api.service.PlatformException
        at com.scalix.wireless.api.service.Connection.execute(Connection.java:132)
        at com.scalix.wireless.api.service.PlatformClient.getMessageChanges(PlatformClient.java:654)
        at com.scalix.wireless.controller.command.SyncCommand.doSync(SyncCommand.java:199)
        at com.scalix.wireless.controller.command.SyncCommand.execute(SyncCommand.java:155)
        at com.scalix.wireless.handlers.WirelessRequestImpl.execute(WirelessRequestImpl.java:176)
        at com.scalix.wireless.HttpRequestHandler.handleRequest(HttpRequestHandler.java:138)
        at com.scalix.wireless.WirelessServlet.doPost(WirelessServlet.java:113)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:647)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
        at org.springframework.web.servlet.mvc.ServletWrappingController.handleRequestInternal(ServletWrappingController.java:158)
        at org.springframework.web.servlet.mvc.AbstractController.handleRequest(AbstractController.java:153)
        at org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter.handle(SimpleControllerHandlerAdapter.java:48)
        at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:875)
        at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:809)
        at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:571)
        at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:511)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:647)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
        at org.springframework.web.filter.AbstractRequestLoggingFilter.doFilterInternal(AbstractRequestLoggingFilter.java:156)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:172)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:174)
        at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:200)
        at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:291)
        at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:775)
        at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:704)
        at org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:897)
        at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689)
        at java.lang.Thread.run(Unknown Source)

So there's been some improvement, but this episode seems far from being over

[resol]

I am also very interested in this. I am thinking of getting a starter pack and would love to synchronize my n900!

[the2nd]

any news on this?

[tomster]

Not really.
All I could find out though is (regarding the scalix logs) is that there's obviously different http-requests sent by my 2 phones in use:

E71:
2010-01-19 14:05:17,974 INFO [WirelessServlet.doPost:103] SERVLET - deviceId: IMEI12345, user: XX - HTTP post Request received: /wirelessUser=XX&DeviceId=IMEI12345&DeviceType=IMEI12345&Cmd=Sync
2010-01-19 14:05:54,099 INFO [SyncCommand.doSync:194] COMMAND - deviceId: IMEI12345, user: XX - executing content sync on folder: Kalender

N900:
2010-01-19 14:19:05,822 INFO [WirelessServlet.doPost:103] SERVLET - deviceId: 54321 - HTTP post Request received: /wirelessCmd=Sync&DeviceId=54321&DeviceType=SmartPhone
2010-01-19 14:19:05,825 INFO [SyncCommand.doSync:194] COMMAND - deviceId: 54321 - executing content sync on folder: Kalender

I have no clue what MS EAS protocols' white papers state but what I noticed right away is the fact that I do miss some sort of user authetification on the N900.
How would the phone or server know what user account to sync? The variable user=XX seems to be missing.
Or is it in later EAS versions that the corresponding IMEI : user account info is stored on the server?

--edit--

BTw that's the way my colleagues iPhone syncs (flawlessly):
iPhone:
2010-01-19 14:38:00,813 INFO [WirelessServlet.doPost:103] SERVLET - deviceId: Appl12345NR, user: YY - HTTP post Request received: /wirelessUser=YY&DeviceId=Appl12345NR&DeviceType=iPhone&Cmd=Sync
2010-01-19 14:38:00,880 INFO [SyncCommand.doSync:194] COMMAND - deviceId: Appl12345NR, user: YY - executing content sync on folder: Calendar

[tomster]

Ok, we're getting closer- naa, if you consider a workaround as fix then we're right there!

Not having any clue of the EAS protocols my assumptions (of course) were completely wrong.
Having a closer look at the N900's log files showed that authetification just works perfect. But a sync-request from the phone on certain folders will create a error output.
So I decided to send the logs of my N900's AS activities to the dev team at Nokia.

This is their response:

"I see some unusual server response to sync command. When we requesting for initial sync keys, server doesn't provide them. Moreover, for some folders, server doesn't provide sync status and, I suspect, doesn't include the necessary information into the response. You know, 12.0 is "not optimized 12.1" and server shall include in the response all collections it received in the request, but I see that it doesn't happen. At the moment we assume that initial sync keys are always provided when requested and, if it is not true, the behavior is wrong at the moment.
Meanwhile, I suggest to force using protocol version 2.5 with that server and see what happens."

I do assume that EAS 12.0/12.1 dectection and/or server responses are not properly implemented in Scalix. Just my 2 cents...

Anyway, I did as suggested and forced the N900 to use the EAS 2.5 protocol. And guess what?! It's working!

Here's how to force the N900 to use EAS 2.5. Please note that you have to follow the exact procedure. If you delete the MfE-account on the phone you have to do it all over again.
First you have to gain root. I don't explain how to do that, because if you don't know you better stay away from the following! I will be not be held liable for data loss or damage of your phone!

Credits to Vitaly Repin from Nokia!!

There is a possibility to force using a particular AS protocol version.
The forcing is currently supported for AS protocol versions 2.5, 12.0 and 12.1. When forcing a version, be sure that Exchange server you are working with supports this version as well. If it doesn't support the forced version, appropriate error banner will be shown.

Below is the instruction how to force using AS version 2.5, but for other versions the only version number needs to be changed.

When configuring the account from wizard, after you entered the data into credentials page, but before autodiscovery, execute the following commands from x-term application:

gconftool-2 -s --type=string /apps/activesync/ActiveSyncAccount1/use_version 2.5
gconftool-2 -s --type=string /apps/activesync/ActiveSyncAccount1Temp/use_version 2.5

then, continue the setup. This will force using protocol version 2.5.

In order to be sure that you correctly forced using AS version 2.5, execute the commands:

sudo gainroot tail -f /var/log/syslog | grep "Created factory"

this command will not finish immediately. Instead, this will listen for syslog and writes an output when AS is selecting the protocol version to use. It shall write an output when:

* credentials are verified
* first sync started
* settings saved (from config applet)

The output shall be: *Created factory for version 2.5* which means that AS is currently working on AS version 2.5

This output is written even if AS logging is turned off.

So, you can keep the x-term window in background and check it from time to time, but the version select is done only in the cases described above.

When you don't need to see this any more, press Ctrl+C to terminate the "tail" command.

When you don't need to use the forcing any more, it is better to delete the account and create it again.

As a side-note you might want to install "mfefolders" from extras-devel repository. With that little tool you can choose which folders to sync (otherwise the phone syncs everything).

[the2nd]

Thanks for this howto. Works great!