TERM PAPER OF SUBJECT:- System Software (CAP: - 658) TOP C:- F!e"-Fast #e"$%a! a&a!

y'er (e&erator
Submitted To Lovely Professional University, Phagwara For the partial fulfillment of the degree of MCA !!! S"M

S)*m$tte+ To:

S)*m$tte+ By:
1

Miss #eha Malhotra 'Le(turer)

#ame$ A%ay &umar *oll #o$ *+,-./a,0 *eg1 #o$,,-.2,-3

ACKNOWLEDGEMENT
!t is not until you underta4e the pro%e(t li4e this one that you reali5e how massive the effort it really is, or how mu(h you must rely upon the Selfless efforts and goodwill of others1 There are many who helped us with this pro%e(t, and we want to than4 them all from the (ore of our 6earts1

7e owe spe(ial words of than4s to our Tea(her M$ss ,e-a Ma!-otra for their vision, thoughtful (ounselling and en(ouragement at every step of the pro%e(t1 7e are also than4ful to the tea(hers of the +epartment for giving us the best of 4nowledge and guidan(e throughout the pro%e(t1

And last but not the least, we find no words to a(4nowledge the Analog (lo(4 appli(ation 8 moral support rendered by our parents in ma4ing the effort a su((ess1 All this has be(ome reality be(ause of their blessings and above all by the gra(e of god1

A.ay /)mar

TAB#E OF CO,TE,TS
TOP C ,AME
PA0E ,O1

Certifi(ate 9999999a +e(laration 9999999 b Abstra(t 9999999 , ,1. !#T*:+UCT!:# 9999999 1.1.0 Introduction to Lexical Grammar 3 1.2.0 Introduction to Token 9999999 ; 1.3.0 How scanner and tokenizer works ? 9999999 ; 1.4.0 lat!orm used 9999999 3 -1. P*:P:S"+ M"T6:+:L:<= 9999999 0 2.1.0 "lock #ia$ram % 2.2.0 #ata &low #ia$ram % 2.3.0 &low '(art 9999999 ,, -1;1. Code A#+ )creens(ot 1111111,21. APP*:AC6"+ *"SULT A#+ C:#CLUS!:# 22 ;1. APPL!CAT!:#S A#+ FUTU*" 7:*& 23 *"F"*"#C"S

ABSTRACT
The le>i(al analy5er is responsible for s(anning the sour(e input file and translating le>emes 'strings) into small ob%e(ts that the (ompiler for a high level language (an easily pro(ess1 These small values are often (alled ?to4ens@1 The le>i(al analy5er is also responsible for (onverting seAuen(es of digits in to their numeri( form as well as pro(essing other literal (onstants, for removing (omments and white spa(es from the sour(e file, and for ta4ing (are of many other me(hani(al details1 Le>i(al analy5er (onverts stream of input (hara(ters into a stream of to4ens1 For to4eni5ing into identifiers and 4eywords we in(orporate a symbol table whi(h initially (onsists of predefined 4eywords1 The to4ens are read from an input file1 The output file will (onsist of all the to4ens present in our input file along with their respe(tive to4en values1 2E34OR5S: Le>eme, Le>i(al Analysis, Compiler, Parser, To4en

617 ,TRO5UCT O,
!n (omputer s(ien(e, !e"$%a! a&a!ys$s is the pro(ess of (onverting a seAuen(e of (hara(ters into a seAuen(e of to4ens1 A program or fun(tion whi(h performs le>i(al analysis is (alled a !e"$%a! a&a!y'er, !e"er or s%a&&er1 A le>er often e>ists as a single fun(tion whi(h is (alled by a parser or another fun(tion and wor4s alongside other (omponents for ma4ing (ompilation of a high level language possible1 This (omplete setup is what we (all a (ompiler1 To define what a (ompiler is one must first define what a translator is1 A translator is a program that ta4es another program written in one language, also 4nown as the sour(e language, and outputs a program written in another language, 4nown as the target language1 #ow that the translator is defined, a (ompiler (an be defined as a translator1 The sour(e language is a high level language su(h as Bava or Pas(al and the target language is a low level language su(h as ma(hine or assembly1 There are five parts of (ompilation 'or phases of the (ompiler) ,1) Le>i(al Analysis -1) Synta> Analysis 21) Semanti( Analysis ;1) Code :ptimi5ation C1) Code <eneration Le>i(al Analysis is the a(t of ta4ing an input sour(e program and outputting a stream of to4ens1 This is done with the S(anner1 The S(anner (an also pla(e identifiers into something (alled the symbol table or pla(e strings into the string table1 The S(anner (an report trivial errors su(h as invalid (hara(ters in the input file1 Synta> Analysis is the a(t of ta4ing the to4en stream from the s(anner and (omparing them against the rules and patterns of the spe(ified language1 Synta>
4

Analysis is done with the Parser1 The Parser produ(es a tree, whi(h (an (ome in many formats, but is referred to as the parse tree1 !t reports errors when the to4ens do not follow the s*ntax of the spe(ified language1 "rrors that the Parser (an report are synta(ti(al errors su(h as missing parenthesis, semi(olons, and 4eywords1 Semanti( Analysis is the a(t of determining whether or not the parse tree is relevant and meaningful1 The output is intermediate (ode, also 4nown as an intermediate representation 'or !*)1 Most of the time, this !* is (losely related to assembly language but it is ma(hine independent1 !ntermediate (ode allows different (ode generators for different ma(hines and promotes abstra(tion and portability from spe(ifi( ma(hine times and languages1 '! dare to say that the most famous e>ample is %avaDs byte (ode and BEM)1 Semanti( Analysis finds more meaningful errors su(h as unde(lared variables, type (ompatibility, and s(ope resolution1 Code :ptimi5ation ma4es the !* more effi(ient1 Code optimi5ation is usually done in a seAuen(e of steps1 Some optimi5ations in(lude (ode hosting, or moving (onstant values to better pla(es within the (ode, redundant (ode dis(overy, and removal of useless (ode1 Code <eneration is the final step in the (ompilation pro(ess1 The input to the Code <enerator is the !* and the output is ma(hine language (ode1

61617 &tro+)%t$o& to #e"$%a! 0rammar

The spe(ifi(ation of a programming language will often in(lude a set of rules whi(h defines the le>er1 These rules are usually (alled regular e>pressions and they define the set of possible (hara(ter seAuen(es that are used to form to4ens or le>emes1 whitespa(e, 'i1e1 (hara(ters that are ignored), are also defined in the regular e>pressions1

1.2.0 Introduction to token

A to/e& is a string of (hara(ters, (ategori5ed a((ording to the rules as a symbol 'e1g1 !+"#T!F!"*, #UMF"*, C:MMA, et(1)1 The pro(ess of forming to4ens from

an input stream of (hara(ters is (alled 'to/e&$'at$o&) and the le>er (ategori5es them a((ording to a symbol type1 A to4en (an loo4 li4e anything that is useful for pro(essing an input te>t stream or te>t file1 A le>i(al analy5er generally does nothing with (ombinations of to4ens, a tas4 left for a parser1 For e>ample, a typi(al le>i(al analy5er re(ogni5es parenthesis as to4ens, but does nothing to ensure that ea(h G'G is mat(hed with a G)G1 Consider this e>pression in the C programming language$ sumH2I-J To4eni5ed in the following table$ To4ens are freAuently defined by regular e>pressions, whi(h are understood by a le>i(al analy5er generator su(h as le>1 The le>i(al analy5er 'either generated
5

automati(ally by a tool li4e le>, or hand (rafted) reads in a stream of (hara(ters, identifies the le>emes in the stream, and (ategori5es them into to4ens1 This is (alled Kto4eni5ing1K !f the le>er finds an invalid to4en, it will report an error1 Following to4eni5ing is parsing1 From there, the interpreted data may be loaded into data stru(tures for general use, interpretation, or (ompiling1

61817 9ow s%a&&er a&+ to/e&$'er wor/:

The first stage, the s%a&&er, is usually based on a finite state ma(hine1 !t has en(oded within it information on the possible seAuen(es of (hara(ters that (an be (ontained within any of the to4ens it handles 'individual instan(es of these (hara(ter seAuen(es are 4nown as le>emes)1 For instan(e, an inte$er to4en may (ontain any seAuen(e of numeri(al digit (hara(ters1 !n many (ases, the first non white spa(e (hara(ter (an be used to dedu(e the 4ind of to4en that follows and subseAuent input (hara(ters are then pro(essed one at a time until rea(hing a (hara(ter that is not in the set of (hara(ters a((eptable for that to4en 'this is 4nown as the ma>imal mun(h rule, or longest mat(h rule)1 !n some languages the le>eme (reation rules are more (ompli(ated and may involve ba(4tra(4ing over previously read (hara(ters1 Tokenization is the pro(ess of demar(ating and possibly (lassifying se(tions of a string of input (hara(ters1 The resulting to4ens are then passed on to some other form of pro(essing1 The pro(ess (an be (onsidered a sub tas4 of parsing input1 Ta4e, for e>ample, the following string1 The Aui(4 brown fo> %umps over the la5y dog Unli4e humans, a (omputer (annot intuitively GseeG that there are 0 words1 To a (omputer this is only a series of ;2 (hara(ters1 A pro(ess of to4eni5ation (ould be used to split the senten(e into word to4ens1 Although the following e>ample is given as LML there are many ways to represent to4eni5ed input$ Msenten(eN MwordNTheMOwordN MwordNAui(4MOwordN MwordNbrownMOwordN MwordNfo>MOwordN MwordN%umpsMOwordN MwordNoverMOwordN MwordNtheMOwordN MwordNla5yMOwordN MwordNdogMOwordN MOsenten(eN A le>eme, however, is only a string of (hara(ters 4nown to be of a (ertain 4ind 'eg, a string literal, a seAuen(e of letters)1 !n order to (onstru(t a to4en, the le>i(al analy5er needs a se(ond stage, the e;a!)ator, whi(h goes over the (hara(ters of the le>eme to produ(e a +alue1 The le>emeGs type (ombined with its value is what properly (onstitutes a to4en, whi(h (an be given to a parser1 'Some to4ens su(h as parentheses do not really have values, and so the evaluator fun(tion for these (an return nothing1 The evaluators for integers, identifiers, and strings (an be (onsiderably more (omple>1 Sometimes evaluators (an suppress a le>eme entirely, (on(ealing it from the parser, whi(h is useful for whitespa(e and (omments1) For e>ample, in the sour(e (ode of a (omputer program the string
6

netPworthPfuture H 'assets liabilities)J might be (onverted 'with whitespa(e suppressed) into the le>i(al to4en stream$

Though it is possible and sometimes ne(essary to write a le>er by hand, le>ers are often generated by automated tools1 These tools generally a((ept regular e>pression that des(ribe the to4ens allowed in the input stream1 "a(h regular e>pression is asso(iated with a produ(tion in the le>i(al grammar of the programming language that evaluates the le>emes mat(hing the regular e>pression1 These tools may generate sour(e (ode that (an be (ompiled and e>e(uted or (onstru(t a state table for a finite state ma(hine 'whi(h is plugged into template (ode for (ompilation and e>e(ution)1 *egular e>pressions (ompa(tly represent patterns that the (hara(ters in le>emes might follow1 For e>ample, for an "nglish based language, a #AM" to4en might be any "nglish alphabeti(al (hara(ter or an unders(ore, followed by any number of instan(es of any ASC!! alphanumeri( (hara(ter or an unders(ore1 This (ould be represented (ompa(tly by the string Qa 5A RPSQa 5A RP. 0ST1 This means Kany (hara(ter a 5, A R or P, followed by . or more of a 5, A R, P or . 0K1 *egular e>pressions and the finite state ma(hines they generate are not powerful enough to handle re(ursive patterns, su(h as Kn opening parentheses, followed by a statement, followed by n (losing parentheses1K They are not (apable of 4eeping (ount, and verifying that n is the same on both sides U unless you have a finite set of permissible values for n1 !t ta4es a full fledged parser to re(ogni5e su(h patterns in their full generality1 A parser (an push parentheses on a sta(4 and then try to pop them off and see if the sta(4 is empty at the end1 The Le> programming tool and its (ompiler is designed to generate (ode for fast le>i(al analysers based on a formal des(ription of the le>i(al synta>1 !t is not generall (onsidered suffi(ient for appli(ations with a (ompli(ated set of le>i(al rules and severe performan(e reAuirementsJ for instan(e, the <#U Compiler Colle(tion uses handwritten le>ers1 61<17 P!atform )se+ !n (omputing, C is a general purpose (omputer programming language originally developed in ,03- by +ennis *it(hie at the Fell Telephone Laboratories to implement the Uni> operating system1 Although C was designed for writing ar(hite(turally independent system software, it is also widely used for developing appli(ation software1 7orldwide, C is the first or se(ond most popular language in terms of number of developer positions or publi(ly available (ode1 !t is widely used on many different software platforms, and there are few (omputer ar(hite(tures for whi(h a C (ompiler does not e>ist1 C has greatly influen(ed many other popular programming languages, most notably CII, whi(h originally began as an e>tension to C, and Bava and CV whi(h borrow C le>i(al (onventions and operators1 '(aracteristics Li4e most imperative languages in the AL<:L tradition, C has fa(ilities for stru(tured programming and allows le>i(al variable s(ope and re(ursion, while a stati( type
7

system prevents many unintended operations1 !n C, all e>e(utable (ode is (ontained within fun(tions1 Fun(tion parameters are always passed by value1 Pass by referen(e is a(hieved in C by e>pli(itly passing pointer values1 6eterogeneous aggregate data types 'stru(4) allow related data elements to be (ombined and manipulated as a unit1 C program sour(e te>t is free format, using the semi(olon as a statement terminator 'not a delimiter)1 C also e>hibits the following more spe(ifi( (hara(teristi(s$ W non nest able fun(tion definitions W variables may be hidden in nested blo(4s W partially wea4 typingJ for instan(e, (hara(ters (an be used as integers W low level a((ess to (omputer memory by (onverting ma(hine addresses to Typed pointers W fun(tion and data pointers supporting ad (oc run time polymorphism W array inde>ing as a se(ondary notion, defined in terms of pointer arithmeti( W a pre pro(essor for ma(ro definition, sour(e (ode file in(lusion, and (onditional Compilation W (omple> fun(tionality su(h as !O:, string manipulation, and mathemati(al Fun(tions (onsistently delegated to library routines W A relatively small set of reserved 4eywords 'originally 2-, now 23 in C00) W A large number of (ompound operators, su(h as IH, II ii. &eatures The relatively low level nature of the language affords the programmer (lose Control over what the (omputer does, while allowing spe(ial tailoring and aggressive :ptimi5ation for a parti(ular platform1 This allows the (ode to run effi(iently on very limited hardware, su(h as embedded systems1 iii. Tur,o '-Turbo CII is a CII (ompiler and integrated development environment '!+") from Forland1 The original Turbo CII produ(t line was put on hold after ,00;, and was revived in -../ as an introdu(tory level !+", essentially a stripped down version of their flagship CII Fuilder1 Turbo CII -../ was released on September C, -../ and is available in G">plorerG and GProfessionalG editions1 The ">plorer edition is free to download and distribute while the Professional edition is a (ommer(ial produ(t1 The professional edition is no longer available for pur(hase from Forland1 Turbo CII 21. was released in ,00, 'shipping on #ovember -.), and (ame in amidst e>pe(tations of the (oming release of Turbo CII for Mi(rosoft 7indows1 !nitially released as an MS +:S (ompiler, 21. supported CII templates, ForlandGs inline assembler, and generation of MS +:S mode e>e(utables for both X.X/ realmode 8 -X/ prote(ted 'as well as the !ntel X.,X/1) 21.Gs implemented AT8T CII -1,, the most re(ent at the time1 The separate Turbo Assembler produ(t was no longer in(luded, but the inline assembler (ould stand in as a redu(ed fun(tionality version1 =17 PROPOSE5 MET9O5O#O03 Aim of the pro%e(t is to develop a Le>i(al Analy5er that (an generate to4ens for the further pro(essing of (ompiler1 The %ob of the le>i(al analy5er is to read the sour(e program one (hara(ter at a time and produ(e as output a stream of to4ens1 The to4ens produ(ed by the le>i(al analy5er serve as input to the ne>t phase, the parser1 Thus, the le>i(al analy5erDs %ob is to translate the sour(e program in to a form more (ondu(tive to re(ognition by the parser1 The goal of this program is to (reate to4ens from the given input stream1 $1 =1617 B!o%/ +$a(ram
8

=1=17 5ata F!ow 5$a(ram A +ata f!ow +$a(ram '5F5) is a graphi(al representation of the KflowK of data through an information system1 !t differs from the flow(hart as it shows the data flow instead of the control flow of the program1 A data flow diagram (an also be used for the visuali5ation of data pro(essing 'stru(tured design)1

=1817 F!ow C-art A flow(hart is a (ommon type of (hart, that represents an algorithm or a pro(ess, showing the steps as bo>es of various 4inds, and their order by (onne(ting these with arrows1 Flow(harts are used in analy5ing, designing, do(umenting or managing a pro(ess or program in various fields1 Flow(harts are used in designing and do(umenting (omple> pro(esses1 Li4e other types of diagram, they help visuali5e what is going on and thereby help the viewer to understand a pro(ess, and perhaps also find flaws, bottlene(4s, and other less obvious features within it1 There are many different types of flow(harts, and ea(h type has its own repertoire of bo>es and notational (onventions1 The two most (ommon types of bo>es in a flow(hart are$ W A pro(essing step, usually (alled acti+it*, and denoted as a re(tangular bo> W A de(ision usually denoted as a diamond1

10

Co+e
using using using using using using using using using using System; System.Collections.Generic; System.ComponentModel; System.Data; System.Drawing; System.Linq; System.Text; System.Windows.Forms; System.T reading; System.!";

namespace Lexical#naly$er % pu&lic partial class 'rmLexical#naly$er ( Form % pu&lic 'rmLexical#naly$er)*

11

% +

!nitiali$eComponent)*;

int count , -; pu&lic .oid display"utput)* // 'or display output in DataGrid0iew % try % #naly$er1ac2Wor2.'ile3 , txt4at .Text; // reading 'ile and store to array #naly$er1ac2Wor2.myMain)*; 'or )int i , -; i 5 #naly$er1ac2Wor2.li&rariesCount; i66* % dg."utput.7ows.#dd)*; dg."utput.7ows8dg."utput.7owCount 9 :;.Cells8<Li&raries<;.0alue , #naly$er1ac2Wor2.li&raries8i;.ToString)*; count66; + 'or )int i , -; i 5 #naly$er1ac2Wor2.2eywordsCount; i66* % dg."utput.7ows.#dd)*; dg."utput.7ows8dg."utput.7owCount 9 count 9 :;.Cells8<7eser.eWords<;.0alue , #naly$er1ac2Wor2.2eywords#rray8i;.ToString)*; + 'or )int i , -; i 5 #naly$er1ac2Wor2.operatorCount; i66* % dg."utput.7ows.#dd)*; dg."utput.7ows8dg."utput.7owCount 9 count 9 #naly$er1ac2Wor2.2eywordsCount 9 :;.Cells8<"perators<;.0alue , #naly$er1ac2Wor2.operators8i;.ToString)*; + 'or )int i , -; i 5 #naly$er1ac2Wor2..arCount; i66* % dg."utput.7ows.#dd)*; dg."utput.7ows8dg."utput.7owCount 9 count 9 #naly$er1ac2Wor2.operatorCount 9 #naly$er1ac2Wor2.2eywordsCount 9 :;.Cells8<0aria&le3ames<;.0alue , #naly$er1ac2Wor2.original0aria&les8i;; + + catc )=xception* %+ + pri.ate .oid &tn"pen>Clic2)o&?ect sender@ =.ent#rgs e* % i' )openFileDialog:.S owDialog)* ,, Dialog7esult."A* % i' )openFileDialog:.File3ame.=ndsWit )<.cpp<.ToBpper)*** % txt4at .Text , openFileDialog:.File3ame;

12

rTextSource.Text , #naly$er1ac2Wor2.readFullFile)txt4at .Text*; + else % Message1ox.S ow)<Must "pen a .cpp 'ile<*; + + else % return; + + pri.ate .oid 'rmLexical#naly$er>Load)o&?ect sender@ =.ent#rgs e* % t is.Si$e , new Si$e)CDE@ FCC*; + pri.ate .oid &tn1ac2>Clic2)o&?ect sender@ =.ent#rgs e* % int widt , :-:D; w ile )widt G, CDE* % t is.Si$e , new Si$e)widt @ FCC*; t is.7e'res )*; t is.SetStyle)ControlStyles.#ll4ainting!nWm4aint@ true*; t is.SetStyle)ControlStyles."ptimi$edDou&le1u''er@ true*; t is.SetStyle)ControlStyles.Bser4aint@ true*; T read.Sleep):-*; widt 9, :-; + + pri.ate .oid &tnGenerate>Clic2)o&?ect sender@ =.ent#rgs e* % // 'or spreading 'orm int widt , CDE; w ile )widt 5, :-:D* % t is.Si$e , new Si$e)widt @ FCC*; t is.7e'res )*; t is.SetStyle)ControlStyles."ptimi$edDou&le1u''er@ true*; t is.SetStyle)ControlStyles.#ll4ainting!nWm4aint@ true*; t is.SetStyle)ControlStyles.Bser4aint@ true*; T read.Sleep):-*; widt 6, :-; + display"utput)*; + pri.ate .oid &tnCancel>Clic2)o&?ect sender@ =.ent#rgs e* % #pplication.=xit)*; +

13

pri.ate .oid dg."utput>CellContentClic2)o&?ect sender@ DataGrid0iewCell=.ent#rgs e* % + pri.ate .oid txt4at >TextC anged)o&?ect sender@ =.ent#rgs e* % + pri.ate .oid rTextSource>TextC anged)o&?ect sender@ =.ent#rgs e* % + + +

Ba%/ wor/ %o+e

using using using using using using System; System.Collections.Generic; System.Linq; System.Text; System.!"; System.Windows.Forms;

namespace Lexical#naly$er % pu&lic class #naly$er1ac2Wor2 % pu&lic static c ar8; To2en , new C ar8C--;; pu&lic static string8; 2eyWords , new String8; % <cout<@ <cin<@ <i'<@ <else<@ <'or<@ <w ile<@ <.oid<@ <int<@ <'loat<@ <c ar<@ <dou&le< +; pu&lic static string8; 2eywords#rray , new String8EC;; pu&lic static int 2eywordsCount , -; pu&lic static string8; operators , new String8EC;; pu&lic static int operatorCount , -; pu&lic static string8; li&raries , new String8EC;; pu&lic static int li&rariesCount , -; pu&lic static c ar8; .aria&les , new C ar8C--;; pu&lic static string8; original0aria&les , new String8EC;; pu&lic static int .arCount , -; pu&lic static int index , -; pu&lic static int cnt , -; pu&lic static string 'ile3; pu&lic static string readFullFile)String 'ile3ame* % try %

14

Text7eader tr , File."penText)'ile3ame*; + catc % + + pu&lic static .oid 7eadFile)* % try % Text7eader tr , File."penText)'ile3*; string line , tr.7eadLine)*; w ile )line H, null* % 'or )int i , -; i 5 line.Lengt ; i66* % To2en8index66; , line8i;; + + + catc % + + line , tr.7eadLine)*; )=xception ex* Message1ox.S ow)ex.Message*; return tr.7eadTo=nd)*; )=xception ex* Message1ox.S ow)ex.Message*;

return null;

pu&lic static String c ar#rrayToString)c ar8; tempStr* % string merged , <<; 'oreac )c ar str in tempStr* % merged 6, str; + + return merged;

pu&lic static .oid extractAeyWords)String toSearc * // searc reser.e words % 'or )int i , -; i 5 2eyWords.Lengt ; i66* % i' )2eyWords8i;.=quals)toSearc ** % 2eywords#rray82eywordsCount66; , toSearc ; // A=IW"7DS ST"7#G= + +

15

+ pu&lic static .oid extract0aria&les#ndDataTypes)C ar8; word* % extractAeyWords)c ar#rrayToString)word**; c ar8; .ar#rr , new C ar8C--;; int .arCounter , -; i' )string.Compare)c ar#rrayToString)word*@ <int<* ,, - JJ string.Compare)c ar#rrayToString)word*@ <c ar<* ,, - JJ string.Compare)c ar#rrayToString)word*@ <'loat<* ,, - JJ string.Compare)c ar#rrayToString)word*@ <dou&le<* ,, -* % w ile )To2en8cnt; H, K;K* .ar#rr8.arCounter66; , To2en8cnt66;; c ar8; temp0aria&le , new C ar8.arCounter;; 'or )int a , -; a 5 .arCounter; a66* temp0aria&le8a; , .ar#rr8a;; 'or )int . , -; . 5 .arCounter; .66* % i' )temp0aria&le8.; ,, K@K* % .aria&les8.arCount; , K K; .arCount , .arCount 6 :; + else i' )temp0aria&le8.; ,, K,K* % w ile )temp0aria&le8.; H, K@K LL . 5 .arCounter 9 :* % + . , . 6 :;

+ else % +

.aria&les8.arCount; , K K; .arCount , .arCount 6 :;

.aria&les8.arCount; , temp0aria&le8.;; .arCount , .arCount 6 :; +

original0aria&les , c ar#rrayToString).aria&les*.Split)new String8; % < < +@ StringSplit"ptions.7emo.e=mpty=ntries*; 'or )int & , -; & 5 .arCounter; &66* % .ar#rr8&; , K K; + .arCounter , -; + +

16

pu&lic static .oid myMain)* % c ar8; word , new C ar8C--;; int counter , -; c ar8; li& , new C ar8C--;; int li&Counter , -; 7eadFile)*; 'or )cnt , -; cnt 5 index; cnt66* % // F"7 CM=CA!3G "4=7#T"7S i' )To2en8cnt; ,, K6K JJ To2en8cnt; ,, K9K JJ To2en8cnt; ,, KNK JJ To2en8cnt; ,, K/K JJ To2en8cnt; ,, KOK JJ To2en8cnt; ,, KLK JJ To2en8cnt; ,, KPK JJ To2en8cnt; ,, K5K JJ To2en8cnt; ,, KGK JJ To2en8cnt; ,, KHK JJ To2en8cnt; ,, K,K* % i' )To2en8cnt; ,, K6K* % i' )To2en8cnt 6 :; ,, K6K* operators8operatorCount66; , To2en8cnt; 6 << To2en8cnt 6 :;; else operators8operatorCount66; , To2en8cnt;.ToString)*; + else i' )To2en8cnt; ,, K9K* % i' )To2en8cnt 6 :; ,, K9K* operators8operatorCount66; , To2en8cnt; 6 << To2en8cnt 6 :;; else operators8operatorCount66; , To2en8cnt;.ToString)*; + else i' )To2en8cnt; ,, K5K* % i' )To2en8cnt 6 :; ,, K,K* operators8operatorCount66; , To2en8cnt; 6 << To2en8cnt 6 :;; else operators8operatorCount66; , To2en8cnt;.ToString)*; + else i' )To2en8cnt; ,, KGK* % i' )To2en8cnt 6 :; ,, K,K* operators8operatorCount66; , To2en8cnt; 6 << To2en8cnt 6 :;; else operators8operatorCount66; , To2en8cnt;.ToString)*; + else i' )To2en8cnt; ,, KHK* %

17

i' )To2en8cnt 6 :; ,, K,K* operators8operatorCount66; , To2en8cnt; 6 << 6 To2en8cnt 6 :;; else operators8operatorCount66; , To2en8cnt;.ToString)*; + else i' )To2en8cnt; ,, K,K* % i' )To2en8cnt 6 :; ,, K,K* operators8operatorCount66; , To2en8cnt; 6 << 6 To2en8cnt 6 :;; else operators8operatorCount66; , To2en8cnt;.ToString)*; + else operators8operatorCount66; , To2en8cnt;.ToString)*; + // F"7 CM=CA!3G L!17#7!=S i' )To2en8cnt; ,, KQK* % w ile )To2en8cnt; H, KGK* li&8li&Counter66; , To2en8cnt66;; li&8li&Counter66; , To2en8cnt;; c ar8; tempLi& , new C ar8li&Counter;; 'or )int a , -; a 5 li&Counter; a66* tempLi&8a; , li&8a;; li&raries8li&rariesCount66; , c ar#rrayToString)tempLi&*; // L!17#7!=S ST"7#G= 'or )int & , -; & 5 li&Counter; &66* % li&8&; , K K; + li&Counter , -;

// F"7 CM=CA!3G 7=ST i' )To2en8cnt; ,, K K JJ To2en8cnt; ,, K)K JJ To2en8cnt; ,, K%K JJ To2en8cnt; ,, K@K JJ To2en8cnt; ,, K5K JJ To2en8cnt; ,, KGK JJ To2en8cnt; ,, K;K JJ To2en8cnt; ,, K+K JJ To2en8cnt; ,, K*K* % word8counter; , KR-K; c ar8; tempWord , new C ar8counter;; 'or )int a , -; a 5 counter; a66* tempWord8a; , word8a;;

18

extract0aria&les#ndDataTypes)tempWord*; // FB3CT!"3 T at extracts .aria&les and datatypes 'or )int ? , -; ? 5 counter; ?66* word8?; , K K; + else % + + + + + counter , -;

word8counter66; , To2en8cnt;;

S%ree& s-ort Start$&( +es$(& form

19

Co+$&( >r$&t

20

O)t>)t s%ree&

Pro(ram $&sert

21

To/e& (e&erate+ from >ro(ram

22

817 APPROAC9E5 RESU#T A,5 CO,C#US O, Le>i(al analysis is a stage in (ompilation of any program1 !n this phase we generate to4ens from the input stream of data1 For performing this tas4 we need a Le>i(al Analy5er1 So we are designing a le>i(al analy5er that will generate to4ens from the given input in a high level language statement1 7e have not used any database for storing the symbol table used in this pro%e(t as su(h, as parsing the entire statement is beyond the s(ope of this pro%e(t1 This ma4es our le>i(al analy5er portable and independent of a +FMS1 This although redu(es the number of 4eywords and spe(ial (hara(ters identifiable by the le>i(al analy5er, and in(reases the length of the (ode, but on the other hand redu(es the program (omple>ity and in(reases the overall speed of the system1 The main features of this le>i(al analy5er (an be summari5ed as$ W Simple implementation1 W Fast le>i(al analysis1 W "ffi(ient resour(e utili5ation1 W Portable <17 APP# CAT O,S A,5 FUTURE 4OR2 This le>i(al analy5er (an be used as a stand alone string analysis tool, whi(h (an analy5e a given set of strings and (he(4 there le>i(al (orre(tness1 This (an also be used to analy5e the string seAuen(es delimited by white spa(es in a C O CII sour(e (ode 'T1( O T1(pp) file and output all the results in a te>t file, if proper fun(tionality of file handling will be used in the sour(e (ode of the le>i(al analy5er, this fun(tionality will not be a part of the present pro%e(t but will be available in an upgraded version, if time permits the development of it1 Further more the appli(ations of a le>i(al analy5er in(lude$ ,1 Te>t "diting -1 Te>t Pro(essing 21 Pattern Mat(hing ;1 File Sear(hing An enhan(ed version of this le>i(al analy5er (an be in(orporated with a Parser having the fun(tionality of synta> dire(ted translation, to ma4e a (omplete Compiler in the future1 The le>i(al assembly of the 4eywords and spe(ial (hara(ters (an be appropriately modified in the sour(e (ode to (reate a new high level language li4e CII1

23

REFERE,CES:
Q,S1 =ashwant &anet4ar, ?Let Us C@, !SF#$ ,.$X, X222 ,/2 3, eighth edition, Pages$ ;-; Y ;231 Q-S1 www1wi4ipedia1org http$OOen1wi4ipedia1orgOwi4iOLe>i(alPanalysis Q2S1 www1dragonboo41stanford1edu http$OOdragonboo41stanford1eduOle(ture notesOStanford CS,;2O.2 Le>i(al Analysis1pdf Q;S1 www1do(store1mi41ua http$OOdo(store1mi41uaOorellyO%avaOlangrefO(h.-P.,1htm QCS1 www1isi1edu http$OOwww1isi1eduOZpedroOTea(hingOCSC!C/C Spring,.OLe(turesOLe>i(alAnalysis1part21/p1pdf Q/S1 www1(s1ber4eley1edu http$OOwww1(s1ber4eley1eduOZhilfingrO(s,/;Opubli(PhtmlOle(turesOnote-1pdf Q3S1 Bohn "1 6op(roft, B1+1 Ullman, ?!ntrodu(tion to Automata Theory, Languages, and Computation@, !SF#$ X, XC.,C 0/ ,, eighteenth edition, Page #o1$ 01 QXS1 Pan4a% Balote, ?An !ntegrated Approa(h To Software "ngineering@, !SF#$ X, 32,0 -3, C, se(ond edition, Pages$ ,2 Y ,31

1111111111111111111111111111111Than4s11111111111111111111111

24