I want to pass two variables to my AWK script from my shell script.
#!/bin/bash
HeaderSize=$(grep -n "# end header data" input.list | tr ":" "\n" | head -n 1)
RecordSize=$(grep -n "# Number of unique" input.list | tr ":" "\n" | tail -n 1 | sed 's/,//g')
echo $HeaderSize
echo $RecordSize
head -n $HeaderSize input.list > temp.list
./parse.awk -v headersize=$HeaderSize -v recordsize=$RecordSize < input.list >> temp.list
File parse.awk looks something like this:
#!/bin/gawk -f
BEGIN {
curline=1;
excludeline=0;
newrecordsize=0;
}
{
if (curline < $headersize) {
curline++;
} else {
if (($2 !~ /test1/) && ($2 !~ /test2/) && ($2 !~ /test3/)) {
print $0;
} else {
excludeline++;
}
}
}
END {
print "#", excludeline, "lines excluded";
newrecordsize = recordsize - excludeline;
printf "# Number of unique items after exclusions: %'d\n", newrecordsize;
}
So I just wanted to copy the header directly over, then do some exclude some of the data.
When I run the Bash file I get an error on the gawk execution:
38
94108
gawk: ./parse.awk:6: fatal: cannot open file `94108' for reading (No such file or directory)
94108 is the value for RecordSize or the second variable I'm trying to pass.
How can I fix this error?
Edit: The answer is found below has to do with double quotes on the variables, which apparently I should be doing everywhere. Here are my current script which works fine.
#!/bin/bash
HeaderSize=$(grep -n "# end header data" input.list | tr ":" "\n" | head -n 1)
RecordSize=$(grep -n "# Number of unique" input.list | tr ":" "\n" | tail -n 1 | sed 's/,//g')
echo "$HeaderSize"
echo "$RecordSize"
head -n "$HeaderSize" input.list > temp.list
./parse.awk -v "headersize=$HeaderSize" -v "recordsize=$RecordSize" < input.list >> temp.list
I also cleaned up the awk script
#!/bin/gawk -f
BEGIN {
excludeline=0;
newrecordsize=0;
}
NR > headersize {
if (($2 !~ /test1/) && ($2 !~ /test2/) && ($2 !~ /test3/)) {
print $0;
} else {
excludeline++;
}
}
END {
print "#", excludeline, "lines excluded";
newrecordsize = recordsize - excludeline;
printf "# Number of unique domains after exclusions: %'d\n", newrecordsize;
}
gawkscript. Something else is at play as your scripts run without error for me."recordsize=$RecordSize", won't do any harm, just putting the quotes around the value in the statement is far more common syntax -recordsize="$RecordSize".