Release 0.15ΒΆ

This release started out in spring 2015 as a relatively simple code cleanup.

I focused on the following problem: re2c used to repeat the whole generation process multiple times. Some parts of the generated program depend on the overall input statistics; they cannot be generated until the whole input has been processed. The usual way is to make stubs for all such things and fix them later. Instead, re2c used to process the whole input, gather statistics, discard the generated output and regenerate it from scratch. Moreover, each generation pass further duplicated certain calculations (for similar reasons). As a result code generation phase was repeated four times.

The problem here is not inefficiency: re2c is fast enough to allow 4x overhead. The real problem is the complexity of code: you have to think of multiple execution layers all the time. Some parts of code are only executed on certain layers and affect each other. Simple reasoning gets really hard.

So the main purpose of this release was to simplify code and make it easier to fix bugs and add new features. However, very soon I realized that some changes in code generation are hard to verify by hand. For example, even a minor rebalancing of if and switch statements may change the generated code significantly. In search of an automatic verification tool I encountered the idea of generating skeleton programs.

Meanwhile I just couldn’t help adding warnings, updating build system and fixing various bugs. A heart-warming consequence of code simplification is that re2c now uses re2c more extensively: not only the main program, but also command-line options, inplace configurations and all kinds of escaped strings are parsed with re2c. l also updated the website (feel free to suggest improvements).

Here is the list of all changes:

  • Updated website http://re2c.org:
    • added examples
    • updated docs
    • added news
    • added web feed (Atom 1.0)
  • Added options:
    • -S, --skeleton
    • --empty-class <match-empty | match-none | error>
  • Added warnings:
    • -W
    • -Werror
    • -W<warning>
    • -Wno-<warning>
    • -Werror-<warning>
    • -Wno-error-<warning>
  • Added individual warnings:
    • -Wundefined-control-flow
    • -Wunreachable-rules
    • -Wcondition-order
    • -Wuseless-escape
    • -Wempty-character-class
    • -Wswapped-range
    • -Wmatch-empty-string
  • Fixed options:
    • -- (interpret remaining arguments as non-options)
  • Deprecated options:
    • -1 --single-pass (single pass is by default now)
  • Reduced size of the generated .dot files.

  • Fixed bugs:
    • #27 re2c crashes reading files containing %{ %} (patch by Rui)
    • #51 default rule doesn’t work in reuse mode
    • #52 eliminate multiple passes
    • #59 bogus yyaccept in -c mode
    • #60 redundant use of YYMARKER
    • #61 empty character class [] matches empty string
    • #115 flex-style named definitions cause ambiguity in re2c grammar
    • #119 -f with -b/-g generates incorrect dispatch on fill labels
    • #116 empty string with non-empty trailing context consumes code units
  • Added test options:
    • -j, -j <N> (run tests in N threads, defaults to the number of CPUs)
    • --wine (test windows builds using wine)
    • --skeleton (generate skeleton programs, compile and execute them)
    • --keep-tmp-files (don’t delete intermediate files for successful tests)
  • Updated build system:
    • support out of source builds
    • support `make distcheck`
    • added `make bootstrap` (rebuild re2c after building with precomplied .re files)
    • added `make tests` (run tests with -j)
    • added `make vtests` (run tests with --valgrind -j)
    • added `make wtests` (run tests with --wine -j 1)
    • added Autoconf tests for CXXFLAGS. By default try the following options: -W -Wall -Wextra -Weffc++ -pedantic -Wformat=2 -Wredundant-decls -Wsuggest-attribute=format -Wconversion -Wsign-conversion -O2 -Weverything), respect user-defined CXXFLAGS
    • support Mingw builds: `configure -host i686-w64-mingw32`
    • structured source files
    • removed old MSVC files
  • Moved development to github (https://github.com/skvadrik/re2c), keep a mirror on sourceforge.