|
Post by n00b on Oct 13, 2016 17:34:44 GMT -6
I think that this function should also have a public variable it can set to let the rest of the program know which keyword it found. Also, I am still working on remaking the excel doc. I will try to post it in the next few days. I have just been so busy the past week that all I have been doing when I am at home is sleeping. I do think your tokenizer is to the point where we can test it with a simple parser. I will post that tonight or tomorrow.
|
|
|
Post by sdlbasic on Oct 17, 2016 6:19:25 GMT -6
That's quite easy, just declare a new global string keyword and then:
in this section if (s==keyWord[i]) { keyword = s; // or keyword = keyWord[i]; return true;}
|
|
|
Post by sdlbasic on Oct 18, 2016 7:20:17 GMT -6
well i did a quick fix in your header file to apply the changes, this is not tested, it was a quick draft.
#ifndef TOKENIZER_H_INCLUDED #define TOKENIZER_H_INCLUDED
#include <iostream> #include <string> #include <cctype> #include <vector>
using namespace std;
vector<string> token; //stores tokens for the current source line void tokens(const std::string &data); //reads current source line and fills token vector int inc(string::size_type &, int); //returns the current position being read on the current source line int iswhite(int); //returns whether the current character is a whitespace bool isLetter(char c); //returns whether or not the current character is a letter bool isDigit(char ch); //returns whether or not the current character is a digit void output_tokens(); //outputs the last set of tokens generated bool isKeyWord(string key);
string keyWords[] = {"MOD", "AND", "NOT", "OR", "XOR"}; string keyword;
void tokens(const std::string &data) { string::size_type x = 0;
while(x < data.length()) { char ch = data[x];
//s_data will hold a number or identifier string s_data = "";
while(isspace(ch)) ch = data[inc(x, 1)];
switch(ch) { case '+': inc(x, 1); token.push_back("<add>"); break; case '-': inc(x, 1); token.push_back("<sub>"); break; case '*': inc(x, 1); token.push_back("<mul>"); break; case '/': inc(x, 1); token.push_back("<div>"); break; case '%': inc(x, 1); token.push_back("<mod>"); case '^': inc(x, 1); token.push_back("<pow>"); break; case '(': inc(x, 1); token.push_back("<par>"); break; case ')': inc(x, 1); token.push_back("</par>"); break; case '.': inc(x, 1); token.push_back("<dot>"); break; case '=': inc(x, 1); token.push_back("<equal>"); break; case '>': ch = data[x+1]; if(ch == '=') { inc(x, 2); token.push_back("<greater_equal>"); } else { inc(x, 1); token.push_back("<greater>"); } break; case '<': ch = data[x+1]; if (ch == '=') { inc(x, 2); token.push_back("<less_equal>"); } else if (ch == '>') { inc(x, 2); token.push_back("<not_equal>"); } else { inc(x, 1); token.push_back("<less>"); } break; case '{': inc(x, 1); token.push_back("<curly>"); break; case '}': inc(x, 1); token.push_back("</curly>"); break; case '[': inc(x, 1); token.push_back("<square>"); break; case ']': inc(x, 1); token.push_back("</square>"); break; case '\"': s_data = "<string>"; ch = data[inc(x, 1)]; while(ch != '\"' && ch != '\r' && ch != '\n') { s_data.push_back(ch); ch = data[inc(x, 1)]; } if ((ch = data[x - 1]) == '\"') token.push_back(s_data); inc(x, 1); break; default: if (isdigit(ch)) { s_data = "<num>"; do { s_data.push_back(ch); ch = data[inc(x, 1)]; } while(isdigit(ch) || ch == '.');
token.push_back(s_data); } else if (isalpha(ch) || ch == '_') { //s_data = "<id>"; do { s_data.push_back(ch); ch = data[inc(x, 1)]; } while(isalnum(ch) || ch == '_');
if(isKeyWord(s_data)) { s_data = "<"+s_data+">"; keyword = s_data; } else { s_data = "<id>"+s_data; } token.push_back(s_data); } break; } } }
int inc(string::size_type &x, int by) { return x = x + by; }
int iswhite(int c) { return (c == ' ' || c == '\t'); }
bool isLetter(char c) { return ((c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z')); }
bool isDigit(char c) { return (c >= '0' && c <= '9'); }
bool isKeyWord(string key) { int len = sizeof(keyWords)/sizeof(keyWords[0]); int i; for(i = 0; i < len; i++) { if(key == keyWords[i]) { return true; } } return false; }
void output_tokens() { for(int i = 0; i < token.size(); i++) { try { cout << token.at(i) << endl; } catch(out_of_range& e) { cout << "Token Out of Range Error: " << e.what() << endl; } } }
#endif // TOKENIZER_H_INCLUDED
|
|
|
Post by n00b on Oct 25, 2016 14:59:57 GMT -6
Thanks for all the work you have put in up to this point. I will probably need another week or 2 to finish the parser. I am redoing it from scratch, and the parser was the most complicated part of rcbasic to begin with so I want to make sure I fix all the errors with the current parser without causing new ones. As soon as I finish this, we can split up the next few jobs. Sorry for not responding in a while but my job has been pretty horrible lately, and my school work hasn't been all that easy on me. I am still dedicated to this project but just like with any hobby projects, life can get in the way.
|
|
|
Post by sdlbasic on Oct 28, 2016 8:48:56 GMT -6
I'm glad that i could help out and waiting for the next one.
|
|
|
Post by sdlbasic on Nov 10, 2016 10:30:26 GMT -6
Hello anybody around here?
|
|
|
Post by n00b on Nov 11, 2016 11:11:43 GMT -6
I'm still here. I just haven't gotten too far along yet. I still have work, school, and other stuff going on. I will post what I have later today.
|
|
|
Post by eyfenna on Dec 24, 2016 16:40:56 GMT -6
Currently investing some time in relearning c++ along a "for dummies" book. Do understand classes and inheritance a bit better now.
Hmm I'll do an exeriment with inheriting token from vector and see where this ends ... I'll post the outcome in case it is reliable and usefull.
|
|
|
Post by n00b on Dec 28, 2016 10:19:42 GMT -6
It will be great to have you on board eyfenna. I know I seem to have abandoned this but I promise I haven't. I am probably going to set aside a 3 to 4 hour block, one day a week to actually work on rcbasic and try to fit in some time to work on my game which is actually coming along quite well. Also sdlBasic admin asked me about how to use my engine with sdlBasic so I will get some info up on that as well. For rcbasic dev, I was going to try to set up a live chat for the dev team. It would be friday or saturday. If anyone is interested just let me know. Also since this is becoming more of a community project, I was thinking of changing the name of rcbasic to something else because the rc in rcbasic is just my initials which was apropriate when it was just me working on it but I feel like it is not a good community project name. If anyone has suggestions on a new name please let me know that as well. I am looking forward to the next branch of this project.
|
|
|
Dev thread
Dec 29, 2016 14:33:23 GMT -6
via mobile
Post by eyfenna on Dec 29, 2016 14:33:23 GMT -6
Inherenting from vector doesn't enable or disables comparing of an enum class instance inside the Object to a Type from the enum class so much on this idea.
|
|
|
Post by transdiv on Jan 1, 2017 14:20:39 GMT -6
Curiosity; The compiler/interpreter is open source?
|
|
|
Post by n00b on Jan 2, 2017 8:48:23 GMT -6
Yes. It is on the github page. Just go to rcbasic.com and go to the "get involved" section.
|
|
|
Post by eyfenna on Jan 3, 2017 13:12:44 GMT -6
Is anything speaking for a combination of Class Token, Class Scanner and vector <Token> Tokens. There's an ENUM Class TokenType that Holds all types of Tokens like nun/string/id/operator/nil, an ENUM Class OperatorType with Stuff like mul/div/Add/.../rparen/lparen/rsquare/lsquare. The class token holds a String, a Double variable a Variable of Type TokenType and a Variable of Type OperatorType....
Currently an idea, can start to code this if there's interest.
|
|
|
Post by sdlbasic on Mar 14, 2017 7:17:44 GMT -6
I was sniffing thru your code and saw that you repeat some const vars that are not changed even if its windows linux or android example:
const int SCREEN_WIDTH = 640; const int SCREEN_HEIGHT = 480;
const int MAX_SCREENS = 16; //32 const int MAX_IMAGES = 32768; //65536; const int MAX_SPRITES = 32; //1024; const int MAX_SOUNDS = 1024; const int MAX_MUSIC = 1; const int MAX_FONTS = 32;
#ifdef RC_ANDROID const int MAX_WINDOWS = 1; #else const int MAX_WINDOWS = 8; #endif
As you can see i change only the const that changes the value the rest is not needed, this way is less code, what do you think if i change some stuff to improve the code?
|
|
|
Post by eyfenna on Oct 1, 2017 1:42:17 GMT -6
Rread an introduction on the topic of immediate and retained ... am I correctly deducting that rcbasic is relying on immediate graphic?
|
|