Org-mode mailing list
 help / color / mirror / Atom feed
* [patch suggestion] Mitigating the poor Emacs performance on huge org files: Do not use overlays for PROPERTY and LOGBOOK drawers
@ 2020-04-24  6:55 Ihor Radchenko
  2020-04-24  8:02 ` Nicolas Goaziou
  0 siblings, 1 reply; 85+ messages in thread
From: Ihor Radchenko @ 2020-04-24  6:55 UTC (permalink / raw)
  To: emacs-orgmode

Emacs becomes very slow when opening and moving around huge org files
with many drawers. I have reported this issue last year in bug-gnu-emacs
[1] and there have been other reports on the same problem in the
internet [2]. You can easily see this problem using the attached file if
you try to move down the lines when all the headings are folded. Moving
a single line down may take over 10 seconds in the file.

According to the reply to my initial emacs bug report [1], the reasons
of performance degradation is huge number of overlays created by org in
the PROPERTY and LOGBOOK drawers. Emacs must loop over all those
overlays every time it calculates where the next visible line is
located. So, one way to improve the performance would be reducing the
number of overlays.

I have been looking into usage of overlays in the org-mode code recently
and tried to redefine org-flag-region to use text properties instead of
overlays:

#+begin_src emacs-lisp
(defun org-flag-region (from to flag spec)
  "Hide or show lines from FROM to TO, according to FLAG.
SPEC is the invisibility spec, as a symbol."
  (pcase spec
    ;; outlines must still use overlays because they rely on
    ;; 'reveal-toggle-invisible feature from reveal.el
    ;; That only works for overlays
    ('outline 
     (remove-overlays from to 'invisible spec)
     ;; Use `front-advance' since text right before to the beginning of
     ;; the overlay belongs to the visible line than to the contents.
     (when flag
       (let ((o (make-overlay from to nil 'front-advance)))
	 (overlay-put o 'evaporate t)
	 (overlay-put o 'invisible spec)
	 (overlay-put o 'isearch-open-invisible #'delete-overlay))))
    (_
     (let ((inhibit-modification-hooks t))
       (remove-text-properties from to '(invisible nil))
       ;; Use `front-advance' since text right before to the beginning of
       ;; the overlay belongs to the visible line than to the contents.
       (when flag
	 (put-text-property from to 'rear-non-sticky t)
	 (put-text-property from to 'front-sticky t)
	 (put-text-property from to 'invisible spec)
         ;; no idea if 'isearch-open-invisible is needed for text
         ;; properties 
	 ;; (overlay-put o 'isearch-open-invisible #'delete-overlay)
	 )))))
#+end_src

To my surprise, the patch did not break org to unusable state and
the performance on the sample org file [3] improved drastically. You can
try by yourself!

However, this did introduce some visual glitches with drawer display.
Though drawers can still be folded/unfolded with <tab>, they are not
folded on org-mode startup for some reason (can be fixed by running
(org-cycle-hide-drawers 'all)). Also, some drawers (or parts of drawers)
are unfolded for no apparent reason sometimes. A blind guess is that it
is something to do with lack of 'isearch-open-invisible, which I am not
sure how to set via text properties.

Any thoughts about the use of text properties or about the patch
suggestion are welcome.  

Best,
Ihor

[1] https://lists.gnu.org/archive/html/bug-gnu-emacs/2019-04/msg01387.html
[2] https://www.reddit.com/r/orgmode/comments/e9p84n/scaling_org_better_to_use_more_medsize_files_or/
[3] See the attached org file in my Emacs bug report: https://lists.gnu.org/archive/html/bug-gnu-emacs/2019-04/txte6kQp35VOm.txt

-- 
Ihor Radchenko,
PhD,
Center for Advancing Materials Performance from the Nanoscale (CAMP-nano)
State Key Laboratory for Mechanical Behavior of Materials, Xi'an Jiaotong University, Xi'an, China
Email: yantar92@gmail.com, ihor_radchenko@alumni.sutd.edu.sg



^ permalink raw reply	[flat|nested] 85+ messages in thread

end of thread, other threads:[~2021-05-03 17:33 UTC | newest]

Thread overview: 85+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-04-24  6:55 [patch suggestion] Mitigating the poor Emacs performance on huge org files: Do not use overlays for PROPERTY and LOGBOOK drawers Ihor Radchenko
2020-04-24  8:02 ` Nicolas Goaziou
2020-04-25  0:29   ` stardiviner
2020-04-26 16:04   ` Ihor Radchenko
2020-05-04 16:56     ` Karl Voit
2020-05-07  7:18       ` Karl Voit
2020-05-09 15:43       ` Ihor Radchenko
2020-05-07 11:04     ` Christian Heinrich
2020-05-09 15:46       ` Ihor Radchenko
2020-05-08 16:38     ` Nicolas Goaziou
2020-05-09 13:58       ` Nicolas Goaziou
2020-05-09 16:22         ` Ihor Radchenko
2020-05-09 17:21           ` Nicolas Goaziou
2020-05-10  5:25             ` Ihor Radchenko
2020-05-10  9:47               ` Nicolas Goaziou
2020-05-10 13:29                 ` Ihor Radchenko
2020-05-10 14:46                   ` Nicolas Goaziou
2020-05-10 16:21                     ` Ihor Radchenko
2020-05-10 16:38                       ` Nicolas Goaziou
2020-05-10 17:08                         ` Ihor Radchenko
2020-05-10 19:38                           ` Nicolas Goaziou
2020-05-09 15:40       ` Ihor Radchenko
2020-05-09 16:30         ` Ihor Radchenko
2020-05-09 17:32           ` Nicolas Goaziou
2020-05-09 18:06             ` Ihor Radchenko
2020-05-10 14:59               ` Nicolas Goaziou
2020-05-10 15:15                 ` Kyle Meyer
2020-05-10 16:30                 ` Ihor Radchenko
2020-05-10 19:32                   ` Nicolas Goaziou
2020-05-12 10:03                     ` Nicolas Goaziou
2020-05-17 15:00                     ` Ihor Radchenko
2020-05-17 15:40                       ` Ihor Radchenko
2020-05-18 14:35                         ` Nicolas Goaziou
2020-05-18 16:52                           ` Ihor Radchenko
2020-05-19 13:07                             ` Nicolas Goaziou
2020-05-23 13:52                               ` Ihor Radchenko
2020-05-23 13:53                                 ` Ihor Radchenko
2020-05-23 15:26                                   ` Ihor Radchenko
2020-05-26  8:33                                 ` Nicolas Goaziou
2020-06-02  9:21                                   ` Ihor Radchenko
2020-06-02  9:23                                     ` Ihor Radchenko
2020-06-02 12:10                                       ` Bastien
2020-06-02 13:12                                         ` Ihor Radchenko
2020-06-02 13:23                                           ` Bastien
2020-06-02 13:30                                             ` Ihor Radchenko
2020-06-02  9:25                                     ` Ihor Radchenko
2020-06-05  7:26                                     ` Nicolas Goaziou
2020-06-05  8:18                                       ` Ihor Radchenko
2020-06-05 13:50                                         ` Nicolas Goaziou
2020-06-08  5:05                                           ` Ihor Radchenko
2020-06-08  5:06                                             ` Ihor Radchenko
2020-06-08  5:08                                             ` Ihor Radchenko
2020-06-10 17:14                                             ` Nicolas Goaziou
2020-06-21  9:52                                               ` Ihor Radchenko
2020-06-21 15:01                                                 ` Nicolas Goaziou
2020-08-11  6:45                                               ` Ihor Radchenko
2020-08-11 23:07                                                 ` Kyle Meyer
2020-08-12  6:29                                                   ` Ihor Radchenko
2020-09-20  5:53                                                     ` Ihor Radchenko
2020-09-20 11:45                                                       ` Kévin Le Gouguec
2020-09-22  9:05                                                         ` Ihor Radchenko
2020-09-22 10:00                                                           ` Ihor Radchenko
2020-09-23  6:16                                                             ` Kévin Le Gouguec
2020-09-23  6:48                                                               ` Ihor Radchenko
2020-09-23  7:09                                                                 ` Bastien
2020-09-23  7:30                                                                   ` Ihor Radchenko
2020-09-24 18:07                                                                 ` Kévin Le Gouguec
2020-09-25  2:16                                                                   ` Ihor Radchenko
2020-12-15 17:38                                                                     ` [9.4] Fixing logbook visibility during isearch Kévin Le Gouguec
2020-12-16  3:15                                                                       ` Ihor Radchenko
2020-12-16 18:05                                                                         ` Kévin Le Gouguec
2020-12-17  3:18                                                                           ` Ihor Radchenko
2020-12-17 14:50                                                                             ` Kévin Le Gouguec
2020-12-18  2:23                                                                               ` Ihor Radchenko
2020-12-24 23:37                                                                                 ` Kévin Le Gouguec
2020-12-25  2:51                                                                                   ` Ihor Radchenko
2020-12-25 10:59                                                                                     ` Kévin Le Gouguec
2020-12-25 12:32                                                                                       ` Ihor Radchenko
2020-12-25 21:35                                                                                     ` Kévin Le Gouguec
2020-12-26  4:14                                                                                       ` Ihor Radchenko
2020-12-26 11:44                                                                                         ` Kévin Le Gouguec
2020-12-26 12:22                                                                                           ` Ihor Radchenko
2020-12-04  5:58                                                       ` [patch suggestion] Mitigating the poor Emacs performance on huge org files: Do not use overlays for PROPERTY and LOGBOOK drawers Ihor Radchenko
2021-03-21  9:09                                                         ` Ihor Radchenko
2021-05-03 17:28                                                           ` Bastien

Org-mode mailing list

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://orgmode.org/list/0 list/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 list list/ https://orgmode.org/list \
		emacs-orgmode@gnu.org
	public-inbox-index list

Example config snippet for mirrors.
Newsgroups are available over NNTP:
	nntp://news.yhetil.org/yhetil.emacs.orgmode
	nntp://news.gmane.io/gmane.emacs.orgmode


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git