0

I am writing a lexer, or tokenizer, for a programming language. One of the main functions is to split the line of source code into "tokens". I am achieving this by splitting on spaces to create string arrays. Therefore, when I want to preserve strings, I must temporarily change the contents to a keyword while the line is split, and then put the strings back in afterwards. This worked until I developed the variable system for the language, and I had to be able to preserve multiple strings. Then all exception hell broke loose.

Exceptions:

NullReferenceException (line 12) subStringArg[ini] = quotes[1];

IndexOutOfRangeException (line 34) value = value.Replace("STRING", subStringArg[ini]);

Minimum reproducible example:

public static string[] LexLine(string line)
        {
            string[] subStringArg = null;
            string[] quotes = null;
            string[] tokens = null;
            int ini = 0; // Random name
            while (line.Contains('\"'))
            {
                if (line.Contains('\"'))
                {
                    quotes = line.Split('\"');
                    subStringArg[ini] = quotes[1];
                }

                if (subStringArg != null)
                {
                    line = line.Replace(quotes[1], "STRING");
                    line = line.Replace("\\", "");
                    line = line.Replace("\"", "");
                }
                ini++;
            }
            tokens = line.Split(' ');
            if (tokens[0] == "Output" && tokens[1] != "STRING")
            {
                tokens[1] = IO.OutputVariable(tokens[1]);
            }
            ini = 0;
            foreach (string i in tokens)
            {
                if (i == "STRING" && subStringArg != null)
                {
                    string value = i;
                    value = value.Replace("STRING", subStringArg[ini]);
                    tokens[currentArrayEntry] = value;
                }
                currentArrayEntry++;
                ini++;
            }
            return tokens;
        }

Source code (from my lang):

Output "Defining variables..." to Console. // Exception thrown here
New string of name "sampleStringVar" with value "sample".
Output "enter your name:" to Console.
Get input.
Output sampleStringVar to Console.

I'm asking here because I am clueless as what to do. I should not be getting a NullReferenceException from assigning values.

4
  • Where you initialize subStringArg? Commented Apr 27, 2020 at 4:20
  • 2
    I think the time has come for you to learn to debug. Watch a couple of video's on YouTube to get the hang of stepping through code. The first error: NullReferenceException is because the subStringArg[ini] hasn't been initialized. The second error IndexOutOfRangeException is because subStringArg[ini] is an array with less items than the ini value. Commented Apr 27, 2020 at 4:23
  • Not where I initialize, where I assign the value of quotes[1] to subStringArg[ini]. also I must clarify that when I debugged, the value of quotes[1] had a value. Commented Apr 27, 2020 at 4:25
  • Jeremy in C# values in an array can be added without being initialized. I also made sure ini = 0 before line 34. When the exception is thrown, quotes[1] is not null, the only null variables are subStringArg and tokens, and ini = 0. Commented Apr 27, 2020 at 4:27

1 Answer 1

3

You set the following

string[] subStringArg = null;

then later you do this

subStringArg[ini] = quotes[1];

But you have not initialised subStringArg so it is still null, so you can't assign anything to it and you will get a NullReferenceError

You must first initialise your array before you can assign anything into it.

Also, you should not assume that you have a value in quotes[1] without first checking it. This will lead to a IndexOutOfRangeException

As another point. The first If statement inside your while loop checks for the same condition as the while loop, so it will always be true!

So the following would be better

string[] subStringArg = new String[enterKnownLengthHere_OrUseALIst];
...
 while (line.Contains('\"'))
 {
        quotes = line.Split('\"');
        if(quotes != null && quotes.Length >= 2)
            subStringArg[ini] = quotes[1];
        else
           //Error handling here!


        if (subStringArg.Length >= 1)
        {
                line = line.Replace(quotes[1], "STRING");
                line = line.Replace("\\", "");
                line = line.Replace("\"", "");
        }
        ini++;
    }
Sign up to request clarification or add additional context in comments.

1 Comment

Nice answer, OP I still highly recommend you learn how to use a debugger, the IDE shows you this. You don't know what you're missing out on!

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.