From mboxrd@z Thu Jan 1 00:00:00 1970 From: David Maus Subject: [test] Mark tests with missing dependencies as "expected to fail" Date: Tue, 18 Oct 2011 06:32:45 +0200 Message-ID: <87fwiqnclu.wl%dmaus@ictsoc.de> Mime-Version: 1.0 (generated by SEMI 1.14.6 - "Maruoka") Content-Type: multipart/signed; boundary="pgp-sign-Multipart_Tue_Oct_18_06:32:45_2011-1"; micalg=pgp-sha256; protocol="application/pgp-signature" Content-Transfer-Encoding: 7bit Return-path: Received: from eggs.gnu.org ([140.186.70.92]:53194) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RG1MA-0004jt-Io for emacs-orgmode@gnu.org; Tue, 18 Oct 2011 00:32:51 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1RG1M9-0007Oq-BW for emacs-orgmode@gnu.org; Tue, 18 Oct 2011 00:32:50 -0400 Received: from app1b.xlhost.de ([213.202.242.162]:52074) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RG1M9-0007Ng-6n for emacs-orgmode@gnu.org; Tue, 18 Oct 2011 00:32:49 -0400 Received: from x60s.ictsoc.de.ictsoc.de (p5485A6F2.dip.t-dialin.net [84.133.166.242]) (Authenticated sender: dmaus@ictsoc.de) by app1b.xlhost.de (Postfix) with ESMTPSA id 082C730484A9 for ; Tue, 18 Oct 2011 06:32:47 +0200 (CEST) List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Sender: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org To: emacs-orgmode@gnu.org --pgp-sign-Multipart_Tue_Oct_18_06:32:45_2011-1 Content-Type: text/plain; charset=US-ASCII Hi all, Currently tests with missing dependency are silently skipped -- it might be worth changing this behavior to not skip them, but mark them as expected to fail. You can do this in ERT by placing the keyword :expected-result followed by either :passed or :failed before the test's body. Benefit of this is that the tests w/ missing dependencies will show up in the ERT result page (with a small letter f) but (obviously) don't count as failures. The following macro is a first shot at a convenient way to define tests with dependencies: #+begin_src emacs-lisp (defmacro org-test-deftest-with-dependency (dependency &rest body) (let* ((docstring (if (stringp (third body)) (list (third body)))) (deftest (nthcdr (if docstring 3 2) body)) (dependency-p (eval dependency))) `(ert-deftest ,@(append (list (first body) (second body)) docstring) ,@(if dependency-p '(:expected-result :passed) '(:expected-result :failed (error "Missing dependency"))) ,@deftest))) #+end_src Here DEPENDENCY is a simple form that evaluates to non-nil if the dependency is met. If marking the tests this way is the agreed way to go we can extend the syntax of a dependency to an alist with a human-readable description of the dependency as key and a list of forms that all must eval to non-nil as value. E.g. #+begin_src emacs-lisp (defvar test-ob-R-dependencies '(("R executable" (org-test-for-executable "R")) ("Package ESS" (featurep 'ess)))) #+end_src And change the expander code to map through this alist and as soon one dependency evals to nil sets a variable `dependency-missing' to the respective dependency. Any comments on this? Best, -- David -- OpenPGP... 0x99ADB83B5A4478E6 Jabber.... dmjena@jabber.org Email..... dmaus@ictsoc.de --pgp-sign-Multipart_Tue_Oct_18_06:32:45_2011-1 Content-Type: application/pgp-signature Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) iF4EABEIAAYFAk6dAW0ACgkQma24O1pEeOY3cQEAtqbtmrQB3IMutqxw4cMD6G78 oNOzXdALq/TWLddAvKgA/in6EMU9qeoGudWhs1n3ZhznqakUNweNBhzX8pUqJX+H =Ur7n -----END PGP SIGNATURE----- --pgp-sign-Multipart_Tue_Oct_18_06:32:45_2011-1--