From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id cO+qMa+dwl7zUAAA0tVLHw (envelope-from ) for ; Mon, 18 May 2020 14:37:35 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id gFxkLa+dwl72XAAAbx9fmQ (envelope-from ) for ; Mon, 18 May 2020 14:37:35 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 40E81940DA4 for ; Mon, 18 May 2020 14:37:35 +0000 (UTC) Received: from localhost ([::1]:40622 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jagtZ-0001x3-PS for larch@yhetil.org; Mon, 18 May 2020 10:37:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42736) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jagrp-0001C6-H5 for emacs-orgmode@gnu.org; Mon, 18 May 2020 10:35:45 -0400 Received: from relay12.mail.gandi.net ([217.70.178.232]:33577) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jagro-0003S4-5O for emacs-orgmode@gnu.org; Mon, 18 May 2020 10:35:44 -0400 Received: from localhost (40-67.ipv4.commingeshautdebit.fr [185.131.40.67]) (Authenticated sender: admin@nicolasgoaziou.fr) by relay12.mail.gandi.net (Postfix) with ESMTPSA id E79D3200010; Mon, 18 May 2020 14:35:37 +0000 (UTC) From: Nicolas Goaziou To: Ihor Radchenko Subject: Re: [patch suggestion] Mitigating the poor Emacs performance on huge org files: Do not use overlays for PROPERTY and LOGBOOK drawers References: <87h7x9e5jo.fsf@localhost> <875zdpia5i.fsf@nicolasgoaziou.fr> <87y2qi8c8w.fsf@localhost> <87r1vu5qmc.fsf@nicolasgoaziou.fr> <87imh5w1zt.fsf@localhost> <87blmxjckl.fsf@localhost> <87y2q13tgs.fsf@nicolasgoaziou.fr> <878si1j83x.fsf@localhost> <87d07bzvhd.fsf@nicolasgoaziou.fr> <87imh34usq.fsf@localhost> <87pnbby49m.fsf@nicolasgoaziou.fr> <87tv0efvyd.fsf@localhost> <874kse1seu.fsf@localhost> Mail-Followup-To: Ihor Radchenko , emacs-orgmode@gnu.org Date: Mon, 18 May 2020 16:35:37 +0200 In-Reply-To: <874kse1seu.fsf@localhost> (Ihor Radchenko's message of "Sun, 17 May 2020 23:40:25 +0800") Message-ID: <87r1vhqpja.fsf@nicolasgoaziou.fr> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain Received-SPF: pass client-ip=217.70.178.232; envelope-from=mail@nicolasgoaziou.fr; helo=relay12.mail.gandi.net X-detected-operating-system: by eggs.gnu.org: First seen = 2020/05/18 10:35:40 X-ACL-Warn: Detected OS = Linux 3.11 and newer X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: emacs-orgmode@gnu.org Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: "Emacs-orgmode" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of emacs-orgmode-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=emacs-orgmode-bounces@gnu.org X-Spam-Score: -1.01 X-TUID: sWkBdvEBUd1+ Hello, Ihor Radchenko writes: > Apparently my previous email was again refused by your mail server (I > tried to add patch as attachment this time). Ah. This is annoying, for you and for me. > The patch is in > https://gist.github.com/yantar92/6447754415457927293acda43a7fcaef Thank you. >> I have finished a seemingly stable implementation of handling changes >> inside drawer and block elements. For now, I did not bother with >> 'modification-hooks and 'insert-in-font/behind-hooks, but simply used >> before/after-change-functions. >> >> The basic idea is saving parsed org-elements before the modification >> (with :begin and :end replaced by markers) and comparing them with the >> versions of the same elements after the modification. >> Any valid org element can be examined in such way by an arbitrary >> function (see org-track-modification-elements) [1]. As you noticed, using Org Element is a no-go, unfortunately. Parsing an element is a O(N) operation by the number of elements before it in a section. In particular, it is not bounded, and not mitigated by a cache. For large documents, it is going to be unbearably slow, too. I don't think the solution is to use combine-after-change-calls either, because even a single call to `org-element-at-point' can be noticeable in a very large section. Such low-level code should avoid using the Element library altogether, except for the initial folding part, which is interactive. If you use modification-hooks and al., you don't need to parse anything, because you can store information as text properties. Therefore, once the modification happens, you already know where you are (or, at least where you were before the change). The ideas I suggested about sensitive parts of elements are worth exploring, IMO. Do you have any issue with them? >> For (2), I have introduced org--property-drawer-modified-re to override >> org-property-drawer-re in relevant *-change-function. This seems to work >> for property drawers. However, I am not sure if similar problem may >> happen in some border cases with ordinary drawers or blocks. I already specified what parts were "sensitive" in a previous message. >> 2. I have noticed that results of org-element-at-point and >> org-element-parse-buffer are not always consistent. `org-element-at-point' is local, `org-element-parse-buffer' is global. They are not equivalent, but is it an issue? Regards, -- Nicolas Goaziou