Release 0.11

• Major incompatible change

  A children parameter has been added to the lambda list of the generic function eclector.parse-result:make-skipped-input-result so that results which represent skipped material can have children. For example, before this change, a eclector.parse-result:read call which encountered the expression #+no-such-feature foo bar potentially constructed parse results for all (recursive) read calls, that is for the whole expression, for no-such-feature, for foo and for bar, but the parse results for no-such-feature and foo could not be attached to a parent parse result and were thus lost. In other words the shape of the parse result tree was

  skipped input result #+no-such-feature foo
  expression result    bar
  

  With this change, the parse results in question can be attached to the parse result which represents the whole #+no-such-feature foo expression so that the entire parse result tree has the following shape

  skipped input result #+no-such-feature foo
    skipped input result no-such-feature
    skipped input result foo
  expression result    bar
  

  Since this is a major incompatible change, we offer the following workaround for clients that must support Eclector versions with and without this change:

  (eval-when (:compile-toplevel :load-toplevel :execute)
    (let* ((generic-function #'eclector.parse-result:make-skipped-input-result)
           (lambda-list      (c2mop:generic-function-lambda-list
                              generic-function)))
      (when (= (length lambda-list) 5)
        (pushnew 'skipped-input-children *features*))))
  (defmethod eclector.parse-result:make-skipped-input-result
      ((client client)
       (stream t)
       (reason t)
       #+PACKAGE-THIS-CODE-IS-READ-IN::skipped-input-children (children t)
       (source t))
    ...
    #+PACKAGE-THIS-CODE-IS-READ-IN::skipped-input-children (use children)
    ...)

  The above code pushes a symbol that is interned in a package under the control of the respective client (as opposed to the KEYWORD package) onto *features* before the second form is read and uses that feature to select either the version with or the version without the children parameter of the method definition. See Maintaining Portable Lisp Programs by Christophe Rhodes for a detailed discussion of this technique.

• The new condition type eclector.reader:state-value-type-error can be used to indicate that a value of an unsuitable type has been provided for a reader state aspect.

• The reader state protocol now provides the generic function (setf eclector.reader:state-value) which allows clients to set reader state aspects in addition to establishing dynamically scoped bindings.

• The macros eclector.reader:unquote and eclector.reader:unquote-splicing now signal sensible errors when used outside of the lexical scope of a eclector.reader:quasiquote macro call. Note that the name of the associated condition type is not exported for now since quasiquotation will be implemented in a separate module in the future.

  Such invalid uses can happen when the above macros are called directly or when the ,, ,@ and ,. reader macros are used in a way that constructs the unquoted expression in one context and then "injects" it into some other context, for example via an object reference #N# or read-time evaluation #.(...). Full example:

  (progn
    (print `(a #1=,(+ 1 2) c))
    (print #1#))

  Another minor aspect of this change is that the condition types eclector.reader:unquote-splicing-in-dotted-list and eclector.reader:unquote-splicing-at-top are no longer subtypes of common-lisp:stream-error. The previous relation did not make sense since errors of those types are signaled during macro expansion.

• Eclector now uses the reader state protocol instead of plain special variables to query and track the legality of quasiquotation operations and the consing dot. The additional reader state aspects are documented but remain internal for now.

  The (internal) macro eclector.reader::with-forbidden-quasiquotation is deprecated as of this release. Clients which really need a replacement immediately can use the new (internal) macro eclector.reader::with-quasiquotation-state.

• Eclector no longer returns incorrect parse results when custom reader macros bypass some reader functionality and the input contains labeled object definitions or references.

  An example of a situation that was previously handled incorrectly is the following

  (defun bypassing-left-parenthesis (stream char)
    (declare (ignore char))
    (loop for peek = (eclector.reader:peek-char t stream t nil t)
          when (eq peek #\))
            do (eclector.reader:read-char stream t nil t)
               (loop-finish)
          collect (let ((function (eclector.readtable:get-macro-character
                                   eclector.reader:*readtable* peek)))
                    (cond (function
                           (eclector.reader:read-char stream t nil t)
                           (funcall function stream peek))
                          (t
                           (eclector.reader:read stream t nil t))))))
  
  (let ((eclector.reader:*readtable* (eclector.readtable:copy-readtable
                                      eclector.reader:*readtable*)))
    (eclector.readtable:set-macro-character
     eclector.reader:*readtable* #\( #'bypassing-left-parenthesis)
    (describe (eclector.parse-result:read-from-string
               (make-instance 'eclector.parse-result.test::simple-result-client)
               "(print (quote #1=(member :floor :ceiling)))")))
  ;; [...]
  ;; Slots with :INSTANCE allocation:
  ;;   %RAW                           = (PRINT '(MEMBER :FLOOR :CEILING))
  ;;   %SOURCE                        = (0 . 43)
  ;; [...]
  ;; The %RAW slot used to contain (MEMBER :FLOOR :CEILING) instead of
  ;; (PRINT '(MEMBER :FLOOR :CEILING)).

• The reader macros for non-decimal radices now accept + in the sign part. For example, Eclector now accepts #x+10 as a spelling of 16.

• The reader macros for non-decimal radices now treat non-terminating macro characters that are valid digits for the respective rational syntax as digits instead of signaling an error. This is in line with the behavior for tokens outside of those reader macros.

  As an example, the following signaled an error before this change:

  (let ((eclector.reader:*readtable*
          (eclector.readtable:copy-readtable eclector.reader:*readtable*)))
    (eclector.readtable:set-macro-character
     eclector.reader:*readtable*
     #\1
     (lambda (stream char)
       (declare (ignore stream char))
       1)
     t) ; non-terminating
    (eclector.reader:read-from-string "#x01"))

• When producing parse results and recovering from an invalid input of a form like

  #1=
  ;; a
  ;; b
  <eof>

  Eclector no longer returns an invalid parse result graph.

• When producing parse results and recovering from an invalid input of a form like #1=#1#, Eclector no longer returns an invalid parse result graph.

• The new generic function eclector.reader:new-value-for-fixup is called by eclector.reader:fixup to compute the replacement value for a labeled object marker, both in ordinary objects and in parse results. Clients can define methods on the new generic function to customize such replacements which is probably only useful when parse results are processed since there is not a lot of leeway in the processing of ordinary objects.

• There is now a default method on eclector.reader:fixup-graph-p which returns true if eclector.reader:labeled-object-state indicates that the labeled object in question is final and circular.

• When eclector.parse-result:parse-result-client is used, eclector.reader:labeled-object-state now returns inner labeled object as its fourth value.

• Elector now breaks up long chains of recursive eclector.reader:fixup calls in order to avoid exhausting available stack space. As a consequence, methods on the generic function eclector.reader:fixup can no longer assume an unbroken chain of recursive calls that correspond to the nesting structure of the object graph that is being fixed up. In particular, a call for an inner object cannot rely on the fact that a particular dynamic environment established by a call for an outer object is still active.