speeding up org (was How could I mix COMMENT and TODO?)

speeding up org (was How could I mix COMMENT and TODO?)

[Rustom Mody]

[-- Attachment #1: Type: text/plain, Size: 1629 bytes --]

François wrote:


> In my very first tries with Org, a few months ago, I put all Org files
> into the agenda, to discover that Org was very, very slow.  So, I
> changed it all and collected all agenda and TODO into three files only,
> holding lots of links to all other Org files where the information
> really was.  Org recovered all its speed.  And besides, to repair the
> lost search capabilities, I kludged M-x rgrep so it could search all Org
> files and "reveal" contents when visiting hits.  Well, the "reveal" does
> not always work, but yet, the quicker search is constantly useful to me.
>
> Currently, having put TODOs back in their proper Org files and declaring
> them as agenda files, 38 agenda files are taken out from 360 Org files.
> Even if slightly less speedy than 3 agenda files, this is still very
> bearable: Org does not crawl.  The way Org handles "org-agenda-files" as
> a string naming a file is really convenient to me, it eases the writing
> of external programs acting on them all.  All in all, very satisfactory!
>


Hi François,

I made the suggestion that ragel should/could be part of emacs:
http://lists.gnu.org/archive/html/emacs-orgmode/2012-03/msg00864.html

Summarised by saying that if ragel is integrated into elisp, org code could
become both significantly faster and more readable.

That most sluggish elisp code may be so due to regular expression code, is
discussed here:
http://lists.gnu.org/archive/html/emacs-devel/2010-04/msg01202.html

[In all fairness this is all a bit OT for an org list and should really be
on an emacs devel list]

Rusi

[-- Attachment #2: Type: text/html, Size: 2066 bytes --]

Re: speeding up org (was How could I mix COMMENT and TODO?)

[Max Mikhanosha]

At Fri, 6 Apr 2012 18:40:43 +0530,
Rustom Mody wrote:
> 
> François wrote:
>  
> 
>     In my very first tries with Org, a few months ago, I put all Org files
>     into the agenda, to discover that Org was very, very slow.  So, I
>     changed it all and collected all agenda and TODO into three files only,
>     holding lots of links to all other Org files where the information
>     really was.  Org recovered all its speed.  And besides, to repair the
>     lost search capabilities, I kludged M-x rgrep so it could search all Org
>     files and "reveal" contents when visiting hits.  Well, the "reveal" does
>     not always work, but yet, the quicker search is constantly useful to me.
>    
>     Currently, having put TODOs back in their proper Org files and declaring
>     them as agenda files, 38 agenda files are taken out from 360 Org files.
>     Even if slightly less speedy than 3 agenda files, this is still very
>     bearable: Org does not crawl.  The way Org handles "org-agenda-files" as
>     a string naming a file is really convenient to me, it eases the writing
>     of external programs acting on them all.  All in all, very satisfactory!
> 
> Hi François,
> 
> I made the suggestion that ragel should/could be part of emacs:
> http://lists.gnu.org/archive/html/emacs-orgmode/2012-03/msg00864.html
> 
> Summarised by saying that if ragel is integrated into elisp, org code could become both significantly faster and more readable.
> 
> That most sluggish elisp code may be so due to regular expression code, is discussed here:
> http://lists.gnu.org/archive/html/emacs-devel/2010-04/msg01202.html
> 
> [In all fairness this is all a bit OT for an org list and should really be on an emacs devel list]

Experiencing slow generation of agenda, was the reason for writing
sticky agendas branch. It takes more of a "fix the symptom" approach,
instead speeding up agenda generation, it caches the agenda buffer
itself and lets user refresh it, while also allowing for multiple
agenda buffers, so that a C-c a / search buffer, does not discard your
C-c a a one.

My initial idea was to wrap all the scanner functions (like
org-todo-list or org-tags-view), with caches, but after a day or so it
was obvious that its not a "a few days" task.

But if someone takes up task of speeding up agenda generation, IMHO
the idea of re-factoring the scanning functions to avoid re-scanning
.org files that had not changed, will have best chance of producing
results.

Regards,
  Max