I have made a small patch implementing the following behavior:

With org-agenda-files = "/home/mfo/org/agenda", a filename.

| Line in agenda-file    | Expands to:                  |
|------------------------+------------------------------|
| $HOME/org/org-mode.org | "/home/mfo/org/org-mode.org" |
| td/td.org              | "/home/mfo/org/td/td.org"    |
| ~/org/test.org         | "/home/mfo/org/test.org"     |
| scratch.org            | "/home/mfo/org/scratch.org"  |
| /home/mfo/org/wep.org  | "/home/mfo/org/wep.org"      |
| ../te.org              | "/home/mfo/te.org"           |
|------------------------+------------------------------|

Here is the patch to current git-head: