org-caldav issue: Search failed: ";\\([A-Za-z0-9-]+\\)="

org-caldav issue: Search failed: ";\\([A-Za-z0-9-]+\\)="

[giles]

org-caldav looks great but I've had a problem on initial sync with
Google.  I'm using the version ending 95629 from Jan 27 to sync with a
corporate Google mail account.

Setup:

#+begin_src emacs-lisp

(setq org-caldav-url "https://www.google.com/calendar/dav")
(setq org-caldav-calendar-id "giles@pexip.com")
(setq org-caldav-inbox "calendar.org")
(setq org-icalendar-timezone "Europe/London")

#+end_src

Then call org-caldav-sync, *Messages* reports:

#+begin_src

Contacting host: www.google.com:443
Getting event 1 of 28
icalendar--read-element: Search failed: ";\\([A-Za-z0-9-]+\\)="

#+end_src
This aborts the sync: rerunning asks if I wish to resume, then fails in
the same manner.

Any suggestions?


-- 
Giles Chamberlin

Re: org-caldav issue: Search failed: ";\\([A-Za-z0-9-]+\\)="

[David Engster]

'giles' writes:
> Contacting host: www.google.com:443
> Getting event 1 of 28
> icalendar--read-element: Search failed: ";\\([A-Za-z0-9-]+\\)="

Could you please do M-x toggle-debug-on-error before running the sync
and post the resulting backtrace here?

-David

Re: org-caldav issue: Search failed: ";\\([A-Za-z0-9-]+\\)="

[giles]

Thanks for the interest.

David Engster <deng@randomsample.de> writes:

> Could you please do M-x toggle-debug-on-error before running the sync
> and post the resulting backtrace here?

Slightly different behaviour this time: first seven events synced fine,
number 8 blew up the same way.

Debugger entered--Lisp error: (search-failed ";\\([A-Za-z0-9-]+\\)=")
  re-search-forward(";\\([A-Za-z0-9-]+\\)=" nil nil)
  icalendar--read-element(VEVENT nil)
  icalendar--read-element(VCALENDAR nil)
  icalendar--read-element(nil nil)
  (let* ((calendar-date-style (quote european)) (ical-list (icalendar--read-element nil nil)) (e (car (icalendar--all-events ical-list))) (zone-map (icalendar--convert-all-timezones ical-list)) (dtstart (icalendar--get-event-property e (quote DTSTART))) (dtstart-zone (icalendar--find-time-zone (icalendar--get-event-property-attributes e (quote DTSTART)) zone-map)) (dtstart-dec (icalendar--decode-isodatetime dtstart nil dtstart-zone)) (start-d (icalendar--datetime-to-diary-date dtstart-dec)) (start-t (icalendar--datetime-to-colontime dtstart-dec)) (dtend (icalendar--get-event-property e (quote DTEND))) (dtend-zone (icalendar--find-time-zone (icalendar--get-event-property-attributes e (quote DTEND)) zone-map)) (dtend-dec (icalendar--decode-isodatetime dtend nil dtend-zone)) (dtend-1-dec (icalendar--decode-isodatetime dtend -1 dtend-zone)) end-d end-1-d end-t (summary (icalendar--convert-string-for-import (or (icalendar--get-event-property e (quote SUMMARY)) "No Title"))) (description (icalendar--convert-string-for-import (or (icalendar--get-event-property e (quote DESCRIPTION)) ""))) (rrule (icalendar--get-event-property e (quote RRULE))) (rdate (icalendar--get-event-property e (quote RDATE))) (duration (icalendar--get-event-property e (quote DURATION)))) (if (and dtstart (string= (cadr (icalendar--get-event-property-attributes e (quote DTSTART))) "DATE")) (setq start-t nil)) (if duration (progn (let ((dtend-dec-d (icalendar--add-decoded-times dtstart-dec (icalendar--decode-isoduration duration))) (dtend-1-dec-d (icalendar--add-decoded-times dtstart-dec (icalendar--decode-isoduration duration t)))) (if (and dtend-dec (not (eq dtend-dec dtend-dec-d))) (message "Inconsistent endtime and duration for %s" summary)) (setq dtend-dec dtend-dec-d) (setq dtend-1-dec dtend-1-dec-d)))) (setq end-d (if dtend-dec (icalendar--datetime-to-diary-date dtend-dec) start-d)) (setq end-1-d (if dtend-1-dec (icalendar--datetime-to-diary-date dtend-1-dec) start-d)) (setq end-t (if (and dtend-dec (not (string= (cadr (icalendar--get-event-property-attributes e ...)) "DATE"))) (icalendar--datetime-to-colontime dtend-dec) start-t)) (list start-d start-t (if end-t end-d end-1-d) end-t summary description))
  org-caldav-convert-event()
  (setq eventdata (org-caldav-convert-event))
  (save-current-buffer (set-buffer (org-caldav-get-event uid)) (goto-char (point-min)) (save-excursion (if (re-search-forward "^SEQUENCE:\\s-*\\([0-9]+\\)" nil t) (progn (org-caldav-event-set-sequence cur (string-to-number (match-string 1)))))) (setq eventdata (org-caldav-convert-event)))
  (while --dolist-tail-- (setq cur (car --dolist-tail--)) (setq uid (car cur)) (setq org-caldav-sync-result (cons (list uid (org-caldav-event-status cur) (quote cal->org)) org-caldav-sync-result)) (setq counter (1+ counter)) (message "Getting event %d of %d" counter (length events)) (save-current-buffer (set-buffer (org-caldav-get-event uid)) (goto-char (point-min)) (save-excursion (if (re-search-forward "^SEQUENCE:\\s-*\\([0-9]+\\)" nil t) (progn (org-caldav-event-set-sequence cur (string-to-number (match-string 1)))))) (setq eventdata (org-caldav-convert-event))) (if (eq (org-caldav-event-status cur) (quote new-in-cal)) (save-current-buffer (set-buffer (find-file-noselect org-caldav-inbox)) (org-caldav-debug-print (format "Event UID %s: New in Cal --> Org inbox." uid)) (goto-char (point-max)) (apply (quote org-caldav-insert-org-entry) (append eventdata (list uid))) (setq buf (current-buffer))) (org-caldav-debug-print (format "Event UID %s: Changed in Cal --> Org" uid)) (let ((marker (org-id-find (car cur) t))) (if (null marker) (progn (error "Could not find UID %s." (car cur)))) (save-current-buffer (set-buffer (marker-buffer marker)) (goto-char (marker-position marker)) (if org-caldav-backup-file (progn (org-caldav-backup-item))) (if (or (eq org-caldav-sync-changes-to-org (quote title-only)) (eq org-caldav-sync-changes-to-org (quote title-and-timestamp))) (progn (org-caldav-change-heading (nth 4 eventdata)))) (if (or (eq org-caldav-sync-changes-to-org (quote timestamp-only)) (eq org-caldav-sync-changes-to-org (quote title-and-timestamp))) (progn (org-caldav-change-timestamp (apply ... ...)))) (if (eq org-caldav-sync-changes-to-org (quote all)) (progn (delete-region (org-entry-beginning-position) (org-entry-end-position)) (apply (quote org-caldav-insert-org-entry) (append eventdata ...)))) (setq buf (current-buffer))))) (org-caldav-event-set-status cur (quote synced)) (save-current-buffer (set-buffer buf) (org-caldav-event-set-md5 cur (md5 (buffer-substring-no-properties (org-entry-beginning-position) (org-entry-end-position))))) (setq --dolist-tail-- (cdr --dolist-tail--)))
  (let ((--dolist-tail-- events) cur) (while --dolist-tail-- (setq cur (car --dolist-tail--)) (setq uid (car cur)) (setq org-caldav-sync-result (cons (list uid (org-caldav-event-status cur) (quote cal->org)) org-caldav-sync-result)) (setq counter (1+ counter)) (message "Getting event %d of %d" counter (length events)) (save-current-buffer (set-buffer (org-caldav-get-event uid)) (goto-char (point-min)) (save-excursion (if (re-search-forward "^SEQUENCE:\\s-*\\([0-9]+\\)" nil t) (progn (org-caldav-event-set-sequence cur (string-to-number ...))))) (setq eventdata (org-caldav-convert-event))) (if (eq (org-caldav-event-status cur) (quote new-in-cal)) (save-current-buffer (set-buffer (find-file-noselect org-caldav-inbox)) (org-caldav-debug-print (format "Event UID %s: New in Cal --> Org inbox." uid)) (goto-char (point-max)) (apply (quote org-caldav-insert-org-entry) (append eventdata (list uid))) (setq buf (current-buffer))) (org-caldav-debug-print (format "Event UID %s: Changed in Cal --> Org" uid)) (let ((marker (org-id-find (car cur) t))) (if (null marker) (progn (error "Could not find UID %s." (car cur)))) (save-current-buffer (set-buffer (marker-buffer marker)) (goto-char (marker-position marker)) (if org-caldav-backup-file (progn (org-caldav-backup-item))) (if (or (eq org-caldav-sync-changes-to-org ...) (eq org-caldav-sync-changes-to-org ...)) (progn (org-caldav-change-heading ...))) (if (or (eq org-caldav-sync-changes-to-org ...) (eq org-caldav-sync-changes-to-org ...)) (progn (org-caldav-change-timestamp ...))) (if (eq org-caldav-sync-changes-to-org (quote all)) (progn (delete-region ... ...) (apply ... ...))) (setq buf (current-buffer))))) (org-caldav-event-set-status cur (quote synced)) (save-current-buffer (set-buffer buf) (org-caldav-event-set-md5 cur (md5 (buffer-substring-no-properties (org-entry-beginning-position) (org-entry-end-position))))) (setq --dolist-tail-- (cdr --dolist-tail--))))
  (progn (let ((--dolist-tail-- events) cur) (while --dolist-tail-- (setq cur (car --dolist-tail--)) (setq uid (car cur)) (setq org-caldav-sync-result (cons (list uid (org-caldav-event-status cur) (quote cal->org)) org-caldav-sync-result)) (setq counter (1+ counter)) (message "Getting event %d of %d" counter (length events)) (save-current-buffer (set-buffer (org-caldav-get-event uid)) (goto-char (point-min)) (save-excursion (if (re-search-forward "^SEQUENCE:\\s-*\\([0-9]+\\)" nil t) (progn (org-caldav-event-set-sequence cur ...)))) (setq eventdata (org-caldav-convert-event))) (if (eq (org-caldav-event-status cur) (quote new-in-cal)) (save-current-buffer (set-buffer (find-file-noselect org-caldav-inbox)) (org-caldav-debug-print (format "Event UID %s: New in Cal --> Org inbox." uid)) (goto-char (point-max)) (apply (quote org-caldav-insert-org-entry) (append eventdata (list uid))) (setq buf (current-buffer))) (org-caldav-debug-print (format "Event UID %s: Changed in Cal --> Org" uid)) (let ((marker (org-id-find ... t))) (if (null marker) (progn (error "Could not find UID %s." ...))) (save-current-buffer (set-buffer (marker-buffer marker)) (goto-char (marker-position marker)) (if org-caldav-backup-file (progn ...)) (if (or ... ...) (progn ...)) (if (or ... ...) (progn ...)) (if (eq org-caldav-sync-changes-to-org ...) (progn ... ...)) (setq buf (current-buffer))))) (org-caldav-event-set-status cur (quote synced)) (save-current-buffer (set-buffer buf) (org-caldav-event-set-md5 cur (md5 (buffer-substring-no-properties (org-entry-beginning-position) (org-entry-end-position))))) (setq --dolist-tail-- (cdr --dolist-tail--)))))
  (let ((events (append (org-caldav-filter-events (quote new-in-cal)) (org-caldav-filter-events (quote changed-in-cal)))) (url-show-status nil) (counter 0) eventdata buf uid) (progn (let ((--dolist-tail-- events) cur) (while --dolist-tail-- (setq cur (car --dolist-tail--)) (setq uid (car cur)) (setq org-caldav-sync-result (cons (list uid (org-caldav-event-status cur) (quote cal->org)) org-caldav-sync-result)) (setq counter (1+ counter)) (message "Getting event %d of %d" counter (length events)) (save-current-buffer (set-buffer (org-caldav-get-event uid)) (goto-char (point-min)) (save-excursion (if (re-search-forward "^SEQUENCE:\\s-*\\([0-9]+\\)" nil t) (progn ...))) (setq eventdata (org-caldav-convert-event))) (if (eq (org-caldav-event-status cur) (quote new-in-cal)) (save-current-buffer (set-buffer (find-file-noselect org-caldav-inbox)) (org-caldav-debug-print (format "Event UID %s: New in Cal --> Org inbox." uid)) (goto-char (point-max)) (apply (quote org-caldav-insert-org-entry) (append eventdata ...)) (setq buf (current-buffer))) (org-caldav-debug-print (format "Event UID %s: Changed in Cal --> Org" uid)) (let ((marker ...)) (if (null marker) (progn ...)) (save-current-buffer (set-buffer ...) (goto-char ...) (if org-caldav-backup-file ...) (if ... ...) (if ... ...) (if ... ...) (setq buf ...)))) (org-caldav-event-set-status cur (quote synced)) (save-current-buffer (set-buffer buf) (org-caldav-event-set-md5 cur (md5 (buffer-substring-no-properties ... ...)))) (setq --dolist-tail-- (cdr --dolist-tail--))))))
  org-caldav-update-events-in-org()
  org-caldav-sync()
  call-interactively(org-caldav-sync record nil)
  command-execute(org-caldav-sync record)
  execute-extended-command(nil "org-caldav-sync")
  call-interactively(execute-extended-command nil nil)

-- 
Giles Chamberlin

Re: org-caldav issue: Search failed: ";\\([A-Za-z0-9-]+\\)="

[giles]

In addition to the back trace I've just posted, one thought springs to
mind given the regex search: My appointments feature a number of Norwegians
whose names contain å, ø: non-ascii characters.  May not be relevant but
...

-- 
Giles Chamberlin

Re: org-caldav issue: Search failed: ";\\([A-Za-z0-9-]+\\)="

[David Engster]

'giles' writes:
> David Engster <deng@randomsample.de> writes:
>> Could you please do M-x toggle-debug-on-error before running the sync
>> and post the resulting backtrace here?
>
> Slightly different behaviour this time: first seven events synced fine,
> number 8 blew up the same way.
>
> Debugger entered--Lisp error: (search-failed ";\\([A-Za-z0-9-]+\\)=")
>   re-search-forward(";\\([A-Za-z0-9-]+\\)=" nil nil)
>   icalendar--read-element(VEVENT nil)
>   icalendar--read-element(VCALENDAR nil)
>   icalendar--read-element(nil nil)

There must be something weird with the events you have. Could be that
it's the special characters you mentioned, but I'm afraid I'll need to
see the event which triggers this.

I just pushed a change to org-caldav which introduces the option for
excessive debug output. Please pull the new version and do

(setq org-caldav-debug-level 2)

before running org-caldav-sync. The iCalendar events will then be put
fully into the *org-caldav-debug-buffer*. Please send me the event which
triggers this bug; it should simply be the last one you see in there.
You can also send me the whole buffer, of course, but *please* remember
to edit/delete any private data from these events you don't want me to
see. I'd also recommend to send it directly to me and not to the list.

-David

Re: org-caldav issue: Search failed: ";\\([A-Za-z0-9-]+\\)="

[David Engster]

David Engster writes:
> 'giles' writes:
>> David Engster <deng@randomsample.de> writes:
>>> Could you please do M-x toggle-debug-on-error before running the sync
>>> and post the resulting backtrace here?
>
>>
>> Slightly different behaviour this time: first seven events synced fine,
>> number 8 blew up the same way.
>>
>> Debugger entered--Lisp error: (search-failed ";\\([A-Za-z0-9-]+\\)=")
>>   re-search-forward(";\\([A-Za-z0-9-]+\\)=" nil nil)
>>   icalendar--read-element(VEVENT nil)
>>   icalendar--read-element(VCALENDAR nil)
>>   icalendar--read-element(nil nil)
>
> There must be something weird with the events you have. Could be that
> it's the special characters you mentioned, but I'm afraid I'll need to
> see the event which triggers this.

Just FYI: Giles send me the offending calendar entry off-list and this
issue is fixed.

-David

Re: org-caldav issue: Search failed: ";\\([A-Za-z0-9-]+\\)="

[giles]

David Engster <deng@randomsample.de> writes:

> Just FYI: Giles send me the offending calendar entry off-list and this
> issue is fixed.

Leaving me as a very happy bunny as I can now further reduce my
excursions outside Emacs.  And my thanks to David for a very rapid turn
around.

-- 
Giles Chamberlin