mks_lexplus -- C++ functions used with MKS lex

SYNOPSIS

typedef unsigned-integral yy_state_t;

BEGIN n;
REJECT
YY_DEBUG
YY_PRESERVE
YY_INTERACTIVE
YYEXIT
LEX_WINDOWS
yymore()

extern HANDLE hInst;

const YYNEWLINE = 10;

class yy_scan {
protected:
	yy_state_t * state;
#ifdef LEX_WINDOWS
	int yylex();
#endif
	...
public:
	char * yytext;
	int yyleng;
	int yylineno;
	FILE *yyin;
	FILE *yyout;
	int yyLexFatal;

	yy_scan(int = 100);
	yy_scan(int, char *, char *, yy_state_t *);
	~yy_scan();

	int yylex();

	virtual int yygetc(void);
	virtual int yywrap(void);
	virtual void yyerror(char *fmt, ...);
	virtual void output(int);
	virtual void YY_FATAL(char *);
	virtual void ECHO(void);

	int input(void);
	int unput(int);
	void setinput(FILE *);
	void setoutput(FILE *);

	void NLSTATE(void);
	void YY_INIT(void);

	void yyless(int);

	void yycomment(char *const delim);
	int yymapch(int delim, int esc);
} ;

DESCRIPTION

Typedefs

yy_state_t: defined by LEX to be the appropriate unsigned-integral for indexing state tables. It is either unsigned char or unsigned int, depending on the size of your scanner.

External Variables

hInst: an externally declared variable which holds the window handle of the application. It is used to access the resources which contain the scanner tables when an application is developed for Microsoft Windows.

Macros

BEGIN n

a macro that you can use in an action to cause LEX to enter a new start condition. See the

LEX Programming Guide

for more information on how to use start conditions.

REJECT

a macro that causes yylex() to discard the current match, and examine the next possible match, if any.

YY_DEBUG

a preprocessor symbol that can be defined when compiling the lex generated code. If defined, yylex() prints a trace of its scanning actions.

YY_PRESERVE

a preprocessor symbol that can be defined when compiling the lex generated code. If defined, yylex() saves the current lookahead. This means that your action code can modify yytext. Defining YY_PRESERVE results in a slower scanner, but it is compatible with UNIX System V lex.

YY_INTERACTIVE

is normally defined as 1. In this case, yylex() attempts to satisfy its input requirements without looking ahead over newlines, which is useful for interactive input. yylex() is slightly faster if YY_INTERACTIVE is defined as 0 (zero). You can do this by adding the following line in the LEX definitions section:

%{
#define YY_INTERACTIVE 0
%}

It is usually not worth changing.

yymore()

is a macro that can be coded in a LEX action to cause yylex() to concatenate the next token to the current token in yytext. yylex() does not re-scan the current token.

YYEXIT

if defined when compiling, YY_FATAL() does not halt the application. Instead, the error code of -2 is returned to the caller of yylex(). YYEXIT is automatically defined when generating a scanner for Microsoft Windows development. See YY_FATAL() for more details.

LEX_WINDOWS

is defined automatically when creating Windows resources to enable the automatic loading and unloading of the resource based tables. This flag should not be enabled manually.

Constants

YYNEWLINE

the value that yylex() uses for newline. Its value reflects that assigned in the LEX translation table. See the

MKS LEX Programming Guide

for more information.

Class yy_scan

The following are members of the class yy_scan, which is defined in the header file produced by lex.

state: a protected member of yy_scan that points to a state table of type yy_state_t. This pointer is initialized in the constructor for yy_scan; see the discussion on Constructors and Destructors that follows.
yytext: a pointer to an array of char containing the current input token recognized by the LEX scanner, and is accessible both within a LEX action and on return of the yylex() function. The token is terminated with a null (zero) byte. yytext is initialized in the constructor for yy_scan; see Constructors and Destructors for more information.
yyleng: the length of the input token in yytext. Your LEX actions should not change yytext after the yyleng offset, since that space is used to store LEX lookahead. If you are porting LEX code that does this, you should see the earlier description of YY_PRESERVE.
yylineno: the current input line number, maintained by LEX.
yyin: determines the input stream for the yylex() and input() functions.
yyout: determines the output stream for the output() function, which processes input that does not match any rules. You can use setinput() and setoutput() to change the values of yyin and yyout. If not explicitly assigned, yyin and yyout are set to the values of stdin and stdout, respectively.
yyLexFatal: a variable which flags if an error has transpired. It is only declared if the macro YYEXIT is defined.

Constructors and Destructors

When a member of class yy_scan is defined, one of two C++ constructors can be used. The usual constructor takes an optional integer argument, which defaults to 100, and allocates tables of that length for yytext and state, using the C++ new operator:

yy_scan(int = 100);

This constructor also sets a flag so that the destructor for yy_scan can delete that space.

The other constructor is defined as:

yy_scan(int sz, char* text, char* save, yy_state_t* s);

This constructor can be used if you want to allocate the state and yytext tables yourself. For example:

#include "lex_yy.hpp"
main()
{
	char text[200];
	yy_state_t state[200];
	yy_scan	scan(200, text, NULL, state);

	scan.yylex();
}

If you use YY_PRESERVE, you must also pass another array, to give room for LEX to copy yytext before a user action; otherwise, the save argument can be NULL.

Your tables should all be of the length specified by sz.

Note:: The destructor for yy_scan does not attempt to delete your tables if you use this constructor.

Virtual Functions

A number of member functions of yy_scan are defined as virtual. This means that you can define your own class derived from yy_scan, and override those member functions with your own definition for that class.

Note:: You must exactly replicate the member function prototype that you are re-defining.

For example, LEX gets input characters by calling the member function yygetc(). This is defined as:

virtual int yygetc(void) { return getc(yyin); }

To define a different yygetc(), define a class derived from yy_scan and provide your own definition of yygetc() for that class; for example:

class my_scan : public yy_scan {
	int	yygetc(void) {	// must match prototype exactly
		// my own input routine
	}
};

Now you can use my_scan exactly like yy_scan, and when you call yylex() its input comes from your new function.

Member Functions

int yylex(void)

is the scanner that LEX defines for you. It scans until your action returns, or until it runs out of input and yywrap() returns 1. yylex() returns the token it locates. A zero or a negative value indicates end of input or an error. When generating a scanner for a Windows environment, yylex() returns a special error code -2 to signify a fatal error has transpired. See YY_FATAL() for more details. YACC-generated parsers call yylex() to get input tokens, so the address of a yy_scan must be passed to the YACC parser.

win_yylex()

when generating output for a Windows environment, LEX renames the standard scanner into a protected member function win_yylex(), and creates a new static member function yylex() as a wrapper which is responsible for loading resource based scanner tables, calling win_yylex() if the load was successful, and unloading the tables when the scan is complete. Thus, the Windows support is in effect transparent to the user of yylex(). Ensure that the global variable hInst is set with the window handle of the application prior to calling yylex() when developing a Windows application with LEX and YACC.

int yygetc(void)

is a virtual function that yylex() calls to obtain input characters. The default definition simply returns the next character from yyin.

int yywrap(void)

is a virtual function that yylex() calls when it gets EOF from yygetc(). The default definition simply returns 1. indicating no more input is available. yylex() then returns 0 indicating end-of-file. If you want to supply more input, you should provide a yywrap() that sets up the new input (possibly by assigning a new file stream to yyin), and then returns 0 to indicate to yylex() that more input is available.

void yyerror(char *,...)

is a virtual function used to print diagnostics. The default definition passes its arguments to vfprintf() with output to the error stream stderr. A newline is written following the message. The current line number, yylineno, is printed if it is non-zero. yyerror() returns after printing its error.

Note:: YACC-generated parsers call yyerror() for diagnostics before error handling (recovery or exit).

void output(int)

is a virtual function that writes its character argument to the LEX output stream yyout.

void YY_FATAL(char *)

is a virtual function that takes a char * message. If the macro YYEXIT is undefined, the message is printed using yyerror(), and the program exits. If it is defined, then the message is printed, and the function yylex() returns to the caller with an error code of -2. Note that yyLexFatal is set true in this function when YYEXIT is defined, thus facilitating the return from yylex().

void ECHO(void)

is a virtual function that prints the matched input token yytext to the LEX output stream yyout.

int input(void)

is a function which returns the next character from the LEX input stream. (This means that LEX does not see it.) This function properly accounts for any lookahead that LEX may be using.

int unput(int)

may be called to make the argument character c the next character to be read by yylex(). The characters are placed in a push-back buffer.

void setinput(FILE *)

is a function that can be used to change the value of yyin.

void setoutput(FILE *)

is a function that can be used to change the value of yyout.

void NLSTATE(void)

a function that resets yylex() as though a newline had been seen on the input.

void YY_INIT(void)

a function that can be used to re-initialize yylex() from an unknown state.

void yyless(int)

a function that shrinks the matched input in yytext to n characters. yylex() re-scans the remaining characters.

Library Routines

C++ users can use the library ROOTDIR/libmks/lex.lib with the Microsoft Visual C++ compiler. For other compilers, you should recreate the LEX library from source. For more detailed information, see the ROOTDIR/libmks/readme.txt file.

The following functions are available in the LEX library.

void yycomment(char * delim)

may be called in a translation when yylex() has matched a sequence of characters which marks the start of a comment. It is given a terminating string, that gives the sequence of characters marking the end of a comment. yycomment() skips over input until it finds this sequence. Newlines found while skipping characters increment yylineno. An unexpected end-of-file produces a suitable diagnostic, using yyerror(). The following lex rules match C++ style comments:

"/*"		yycomment("*/");
//.*\n		;

Note:: A LEX pattern is more efficient at recognizing a newline-terminated comment, while the yycomment() function can handle comments longer than the size of yytext.

int yymapch(int delim, int esc)

may be used to process C or C++ style character constants or strings. It reads the next string character from the input, given the required delimiter and escape characters; if the delim is reached, -1 is returned. The usual escapes are recognized; esc is the escape character to use: for C it would be backslash. The contents of yytext are not altered.

Here is an example of a LEX action that reads C and C++ style strings:

\"	{
	int ch; char s[100]; char * cp = s;
	while ((ch = yymapch(*yytext, '\\'))!=-1)
		*cp++ = ch;
	*cp = '\0';
	cout << "string =>" << s << "<=\n";
    }

FILES

ROOTDIR/libmks/*.cpp: C++ source files for the LEX libraries
ROOTDIR/libmks/*.lib: Microsoft Visual C++ libraries

AVAILABILITY

PTC MKS Toolkit for Developers
PTC MKS Toolkit for Interoperability

mks_lexplus
C++ functions used with MKS lex

Miscellaneous Information