Bug #95

fixes in TextMappers.Scanner with option 'interpretSets'

Added by I. Denisov about 3 years ago. Updated almost 3 years ago.

Status:ClosedStart date:01/15/2016
Priority:NormalDue date:
Assignee:R. Campbell% Done:

100%

Category:-
Target version:1.7
Forum topic:

Description

Scanning sets needs the following fixes:

1. checks for missing or duplicate commas
2. the start position should be set to the opening "{"
3. incomplete sets that are terminated by eot (end of text) should always yield 'invalid'.

Associated revisions

Revision acf3cc24
Added by I. Denisov about 3 years ago

Improving SETs scanner. Refs: #95

Revision 41490688
Added by I. Denisov about 3 years ago

The code of Set procedure in TextMappers was made more readable. Refs: #95

Revision 5336fc5d
Added by J. Templ almost 3 years ago

fixes of Set based on BB1.6. Refs: #95.
- no state machine involved, changes are based on BB1.6 version
- s.start fixed: set to the position of the opening "{"
- s.type set to invalid in case of end-of-text within a set
- Skip before "}" removed because it is redundant

Signed-off-by: Josef Templ <>

History

#1 Updated by I. Denisov about 3 years ago

Robert suggested the solution:

       PROCEDURE Sett (VAR s: Scanner);
          VAR n, m, state: INTEGER; ch: CHAR;
       BEGIN
          s.type := set; Get(s, ch); s.Skip(ch); s.set := {}; state := 0;
          LOOP
             CASE ch OF
                ',' : IF state  IN {1, 3} THEN Get(s, ch); s.Skip(ch); state := 4
                      ELSE s.type := invalid; EXIT END
             |   '.' : IF  state # 1 THEN s.type := invalid; EXIT END;
                   Get(s, ch); IF ch # '.' THEN s.type := invalid; EXIT END;
                   Get(s, ch); s.Skip(ch); state  :=  2
             |   '}' : IF state IN {0, 1, 3} THEN Get(s, ch) ELSE s.type := invalid END; EXIT
             |   '0'..'9' : IF ODD (state) THEN s.type := invalid; EXIT END;
                   Cardinal(s,m); s.Skip(ch);
                   IF m > MAX(SET) THEN s.type := invalid; EXIT END;
                   IF state = 2 THEN
                      IF m  < n THEN s.type := invalid; EXIT END;
                      WHILE m > n DO INCL(s.set, m); DEC(m) END;
                      state := 3
                   ELSE
                      INCL(s.set, m); n := m; state := 1
                   END
             ELSE
                s.type := invalid; EXIT
             END
          END
       END Sett;

and test:

    MODULE TestScanner;

    (*  Robert Campbell,  *)

       IMPORT
          TextMappers, TextModels, Log := StdLog;

    PROCEDURE  Do*;
       VAR
          text: TextModels.Model;
          fmtr: TextMappers.Formatter;
          sc: TextMappers.Scanner;
       BEGIN
          text := TextModels.dir.New();
          fmtr.ConnectTo(text);
          fmtr.WriteString ('{1, 13, 17, 22}'); fmtr.WriteLn;
          fmtr.WriteString ('{2, 13..17, 22}'); fmtr.WriteLn;
          fmtr.WriteString ('{3, 13, 17 22}'); fmtr.WriteLn;
          fmtr.WriteString ('{4, 13, 17, 22,}'); fmtr.WriteLn;
          fmtr.WriteString ('{5, 13, 17, A}'); fmtr.WriteLn;
          fmtr.WriteString ('{6}'); fmtr.WriteLn;
          fmtr.WriteString ('{}'); fmtr.WriteLn;

          sc.ConnectTo (text);
          sc.SetOpts ({TextMappers.interpretSets});
          sc.Scan;
          WHILE sc.type # TextMappers.eot DO
             CASE sc.type OF
                TextMappers.char : Log.String('Character : '); Log.Char(sc.char); Log.Ln
             |   TextMappers.string : Log.String('String : ' + sc.string); Log.Ln
             |   TextMappers.int : Log.String('Integer :  ' ); Log.Int(sc.int); Log.Ln
             |   TextMappers.set : Log.Set(sc.set); Log.Ln
             |   TextMappers.invalid : Log.String('Invalid ! '); Log.Ln
             END;
             sc.Scan;
          END
       END Do;

       END TestScanner.

       DevDebug.Unload

      TestScanner.Do

    OMS Scanner:
     {1, 13, 17, 22}
     {2, 13..17, 22}
     {3, 13, 17, 22}
     {4, 13, 17, 22}
    Invalid !
    String : A
    Character : }
     {6}
     {}

    Suggested modified Scanner:
     {1, 13, 17, 22}
     {2, 13..17, 22}
    Invalid !
    Integer :   22
    Character : }
    Invalid !
    Character : }
    Invalid !
    String : A
    Character : }
     {6}
     {}

#2 Updated by I. Denisov about 3 years ago

I changed LOOP to WHILE


    PROCEDURE Set (VAR s: Scanner);
        VAR n, m, state: INTEGER; ch: CHAR;
    BEGIN
        s.type := set; Get(s, ch); s.Skip(ch); s.set := {};
        state := 0;
        WHILE (s.type # invalid) & (ch # '}') DO
            CASE ch OF
            | ',' :
                IF state IN {1, 3} THEN Get(s, ch); s.Skip(ch); state := 4 ELSE s.type := invalid END
            | '.' :
                IF state # 1 THEN s.type := invalid
                ELSE
                    Get(s, ch);
                    IF ch # '.' THEN s.type := invalid ELSE Get(s, ch); s.Skip(ch); state := 2 END
                END
            | '0'..'9' :
                IF ODD(state) THEN s.type := invalid
                ELSE
                    Cardinal(s,m); s.Skip(ch);
                    IF m > MAX(SET) THEN s.type := invalid
                    ELSE
                        IF state = 2 THEN
                            IF m < n THEN s.type := invalid
                            ELSE WHILE m > n DO INCL(s.set, m); DEC(m) END; state := 3 END
                        ELSE INCL(s.set, m); n := m; state := 1 END
                    END
                END
            ELSE s.type := invalid
            END
        END;
        IF s.type = set THEN
            IF (ch = '}') & (state IN {0, 1, 3}) THEN Get(s, ch) ELSE s.type := invalid END
        END
    END Set;

#3 Updated by J. Templ almost 3 years ago

  • Subject changed from TextMapper Scanner & SETS to fixes in TextMappers.Scanner with option 'interpretSets'
  • Description updated (diff)

#4 Updated by I. Denisov almost 3 years ago

  • Status changed from In Progress to Closed
  • % Done changed from 50 to 100

Also available in: Atom PDF