When I tried to load an exported ICS file into Baikal’s calendar I encountered fatal problems. Most of the appointments could not be loaded. A similar problem occurred in conjunction with ownCloud’s calendar. The uploaded ICS file did not follow the icalendar/vcard format, ownCloud reported in its log. I found out that the problematic events in the ICS file were containing properties brought in by invitations generated by Lotus Notes. The reason why Baikal and ownClound could not deal with them was finally found easily. Both products, ownCloud and Baikal, are using SabreDAV for implementing their CalDAV functionality. SabreDAV applies strictly the iCalendar specification (RFC 2445) while parsing ICS files. Unfortunately extensions by Notes or Outlook might be problematic. This default behavior of “strict applying of the specification” can be changed. A simple change of a default parameter setting worked fine for me in Baikal as well as in ownCloud.
Situation A: Baikal
A while ago I had some problems with Thunderbird/Lightning and Google’s calendar. Lightning told me regularly that it was not able to connect to Google. Google’s calendar did a good job for a long time but this incompatibility with Lightning was bothersome. So after a while of suffering it was time for a change. Baikal looked fine. Lightweight, easy to install and maintain.
I exported my Google calendar by Lightning’s export function to an iCalendar file (ICS). Then I connected a fresh installed Baikal 0.2.5 and tried to import. Unfortunately many events were not imported. I realized the ignored events were invitations sent from customers using Lotus Notes…
Situation B: ownCloud
Now I gave ownCloud a chance because a web calendar and the possibility to share calendars looked really smart. I loaded up a similar exported ICS to a freshly installed ownCloud 6. Every time when I tried to click on the ICS file the promised import dialog did not appear. Instead I found error messages like this in ownCloud’s log:
Error vobject Invalid VObject, line 4192 did not follow the icalendar/vcard format
Again I had to realize it’s the old problem with events resulting from Lotus Notes’ invitations…
Extensions to iCalendar?
I started experimenting with the X-LOTUS-* lines in the ICS file. When I removed these lines all could be imported without any problems. OK, so far so good. That’s the import.
In the case of Baikal as well as in the case of ownCloud the problem was not only the initial import but also later received invitations. Editing such an invitation an manually loading it solved the problem here, too. But hang on! Did I want to edit every invitation by removing X-LOTUS-* lines? Not really!
And what if some other groupware or mail client might send me events with other “extensions” likes Notes did with its X-LOTUS-* lines?
It was time to have a look at the iCalendar code of Baikal and ownCloud. Surprisingly both products are using the same library for parsing ICS files — SabreDAV. The bad news: By default SabreDAV is very strict at parsing iCalendar files. It does not tolerate “extensions” not following the rules. The good news: You can change this default behavior to turn a blind eye to such “extensions”.
Problem: Reader.php of SabreDAV
Let’s have a look at Reader.php
of SabreDAV. You will find it in Baikal as Core/Frameworks/SabreDAV/vendor/sabre/vobject/lib/Sabre/VObject/Reader.php
and in ownCloud as 3rdparty/Sabre/VObject/Reader.php
. At or near line 44 you will find method read()
which looks like this:
static function read($data, $options = 0) {
As you will have noticed the default value of parameter $options
is 0. Unless something different is set or given this means apply the iCalendar specification strictly. If you change this default value to 2 (OPTION_IGNORE_INVALID_LINES
) any line which cannot be parsed by the reader will be ignored. Note that it is ignored by the parser only. The line will be still stored in the calendar’s event; in both cases — in Baikal as well as in ownCloud.
To fix the problems described above you simply change the read()
function to this:
static function read($data, $options = 2) {
In both cases, Baikal and ownCloud, I was able to load the ICS file without any problems afterwards. Even later events received within invitations could be stored in both calendars without any further problem.
Conclusion
This solution worked just fine with Baikal for me for over a year now. Until now in ownCloud I did not experience any problems. It seems to work as good as in Baikal but I did not use it for a long time. If I experience any problems I will update this post here.
The only problem with this workaround is whenever I upgrade Baikal or ownCloud I have to edit Reader.php
again. But that’s a little price for a free calendar under me own control.
Note: You cannot blame Lotus Notes for this problem. First of all, if my customers were not using Notes I could have discovered similar problems with Outlook/Exchange or any other mail and groupware system, too. Second, systems like Notes and Domino respectively depend on “extensions” mentioned above. On the other side SabreDAV’s strategy to use by default a strict approach is absolutely right, too. It cares in this case about “vanilla iCalendar” only. The two approaches doesn’t interoperate simply.
For short: It’s simply an incompatiblity. For such situations parameters were invented, weren’t they?
Hi!
I ran into your blog when looking at referrals. I’m the author for sabre/vobject and sabre/dav.
Vobject has no problem with extensions (X- properties). This is not considered an ‘invalid line’. The only case this error would be triggered, is if there’s for instance incorrect characters in the iCalendar object, such as control characters (0x00), or other characters that are not allowed in property names. (for instance, an underscore is not ok in a property name, but is allowed in it’s value).
It’s a bad idea for sabredav to change the default. The problem is, when you allow these invalid objects to be stored in the database, it will also be sent back to other caldav clients. If those clients have conforming iCalendar parsers, they will also break, and often stop syncing.
This could cause data-loss for your users down the line.
It’s better to be strict and reject these. If you provide some importer somewhere, you *could* first run vobject’s repair function to fix these broken objects.
The repair function isn’t perfect, but it can find a bunch of common problems.
Evert
Thanks for your reply. I agree with you that it is not a clean solution.
Nevertheless it’s a quick fix which solved my problem with ownCloud’s calendar. To be strict and reject invitations from Outlook or Lotus Notes is not an option to me. It would make ownCloud useless to me. My customers use intensively Lotus Notes and Outlook. I need to get these invitations into my calendar.
To be strict and rejecting non-standard object is right from a theoretically point of view. In the real world I need working solutions–even if the data (in this case: invitations) are transferred in a format not conforming to a strict standard. You will need to process these data, too.
For me it is a question of being pragmatic. I need a working calendar. This fix provides it to me. I can sync my calendar with my calendar on Mac OSX, with CalDAV-Sync on several Android devices (HTC, Samsung, Acer, and others), and with Thunderbird/Lightning on Mac, Windows, and Linux. It works fine for nearly two years now.
BTW: If it is not a good idea to change the default behavior of SabreDAV. Why does function read offers the option parameter? – Is it just for testing purposes or shall it be an option to take in real world applications?
If you want to have a look at the problematic invitations I will be happy to provide you with samples of these problematic invitations.
Looking forward to hearing from you.
Kind regards,
Oliver
PS: I do not blame SabreDAV as you can read at the end of my posting: “On the other side SabreDAV’s strategy to use by default a strict approach is absolutely right, too.” 😉
Pingback: Android : ne plus utiliser google/gmail pour le carnet d'adresse - Stramanari
Thank you very much for this analysis and solution. Really great.
I changed the default parameter in the Sabre lib in my Own cloud and my daughter can impory their university calendar. 🙂 Syncing with Thunderbird and Android works, so I see no bad side effect.
To be strict with formats may be formally OK, but in practice it seldom meets the reality.
Thanks again
Ralf