0

(I have made several edits to this in response to comments)

My goal is to have a shell script that goes to an external server, gets a batch of files from a directory, prints them on a thermal receipt printer, then deletes the same files from the remote server (so that when a cron job runs they don't get printed again). My problem is in the concatenation of a variable to the end of the URL for cURL. The rest of the script is working but I will show the entire script for clarity

I've done several searches for solutions and many seem to involve more complex situations, e.g. this one that tries to solve for a hidden carriage return since the variable is appended to the middle of the URL (c.f. Bash curl and variable in the middle of the url). I tried that solution (and several others) and they didn't work. My situation is simpler (I think) so maybe those answers added unnecessary complications and that's my problem? Anyways...

Here's the full code with placeholders for sensitive info:

!/bin/bash
# step 1 - change to the directory where we want to temporarily store tickets
cd Tickets
# step 2 - get all the tickets in the target directory on the external server and put them in the current temporary local directory
wget --secure-protocol TLSv1_2 --user=<placeholder> --password='<placeholder>d' ftps://<placeholder>/public_html/tickets/*.txt
# step 3 - print each of the tickets in the current temporary local directory
for i in *.txt
do lp $i
done
# step 4 - reach out to the target directory and delete each of the files that we previously downloaded (not the entire directory; might have new files)
for i in *.txt
do curl --verbose --ftp-ssl --user <placeholder>:<placeholder> 'ftp://<placeholder>/public_html/tickets' -Q "DELE /public_html/tickets/$i"
done
# empty the current local directory where we temporarily stored files during the execution of this script
for i in *.txt
do rm $i
done
# should be done now.

I have used all of the following variations for step 4:

for i in *.txt
do curl --ftp-ssl --user (myftpid):(mypasswd) ftp://(myhostname)/public_html/tickets/ -Q 'DELE /public_html/tickets/'$i
done

for i in *.txt
do curl --ftp-ssl --user (myftpid):(mypasswd) ftp://(myhostname)/public_html/tickets/ -Q 'DELE /public_html/tickets/'${i}
done

for i in *.txt
do curl --ftp-ssl --user (myftpid):(mypasswd) ftp://(myhostname)/public_html/tickets/ -Q 'DELE /public_html/tickets/'"$i"
done

for i in *.txt
do curl --ftp-ssl --user (myftpid):(mypasswd) ftp://(myhostname)/public_html/tickets/ -Q 'DELE /public_html/tickets/'"${i}"
done

Output for all four of those is:

curl: (21) QUOT command filed with 550

I was able to confirm that the code works without a variable by testing this:

curl --ftp-ssl --user <placeholder>:<placeholder> ftp://<placeholder>/public_html/tickets/ -Q 'DELE /public_html/tickets/14.txt'

*** EDIT ** I re-read the comments and I think I initially misunderstood some of them. I was able to use echo in front of the curl command to see the output with the variable. This was very helpful, thanks @Bodo. The suggestion from @NationBoneless for the verbose tag was also useful and yielded the following:

< 220-You are user number 1 of 50 allowed.
< 220-Local time is now 18:34. Server port: 21.
< 220-This is a private system - No anonymous login
< 220-IPv6 connections are also welcome on this server.
< 220 You will be disconnected after 30 minutes of inactivity.
AUTH SSL
< 500 This security scheme is not implemented
AUTH TLS
< 234 AUTH TLS OK.
* successfully set certificate verify locations:
*   CAfile: none
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / <placeholder>
* Server certificate:
*  subject: CN=<placeholder>
*  start date: Nov 16 00:00:00 2020 GMT
*  expire date: Nov 16 23:59:59 2021 GMT
*  subjectAltName: host "<placeholder>" matched cert's "<placeholder>"
*  issuer: C=US; ST=TX; L=Houston; O=cPanel, Inc.; CN=cPanel, Inc. Certification Authority
*  SSL certificate verify ok.
USER <placeholder>
< 331 User <placeholder> OK. Password required
PASS <placeholder>
< 230 OK. Current restricted directory is /
PBSZ 0
< 200 PBSZ=0
PROT P
< 200 Data protection level set to "private"
PWD
< 257 "/" is your current location
* Entry path is '/'
DELE /public_html/tickets/14.txt
* ftp_perform ends with SECONDARY: 0
< 550 Could not delete /public_html/tickets/14.txt: No such file or directory
* QUOT command failed with 550
* Closing connection 0
Improve this question
5
  • 1
    Please edit your question and add more details: What exactly means "none have worked"? Show the exact command and the corresponding output/result/error message. To see what's going on you could add set -x in your script to get the commands printed or insert echo before curl, i.e. something like for i in *.txt; do echo curl --ftp-ssl ... Commented Jan 19, 2021 at 15:35
  • @Bodo - I have made the requested edit. Commented Jan 19, 2021 at 16:34
  • 1
    It does not help much to show a short snippet of your code in different variants. Show every individual full command and the corresponding result. You still did not make clear what exactly "none have worked" means. In which way it does not work? Do you get an error message? An unexpected result? No result at all? Try the two proposed ways to show the commands that will/would be executed. If you don't see the problem with this, add the output to the question. Commented Jan 19, 2021 at 16:44
  • @Bodo - I have made the requested edits. Commented Jan 19, 2021 at 17:24
  • The verbose output shows DELE /public_html/tickets/14.txt followed by 550 Could not delete /public_html/tickets/14.txt: No such file or directory. The DELE protocol command looks the same as I would expect for your curl command with a hard coded file name. Did the file exist at this time? If yes, maybe there are invisible characters somewhere. Did you copy&paste the code and/or output between your terminal and the web browser? According to the typo in the output for all four cases you might have re-typed it which may introduce or hide errors. Commented Jan 20, 2021 at 14:12

2 Answers 2

Reset to default
1

I would think something like this would work. When I tested it locally the curl command seemed to work correctly:

for i in *.txt
do curl --ftp-ssl --user (myftpid):(mypasswd) ftp://(myhostname)/public_html/tickets/ -Q "DELE /public_html/tickets/$i"
done
Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

I just tested that format. It gave the same error as before. Any other ideas?
And are you certain, that all the files *.txt expands to in that directory are also on the ftp server? Also, you could add the --verbose flag to your curl command, to get more information about the execution. If you could add the information from the command with the verbose flag, that would also help pinpoint the error
I'm certain the files are there. The --verbose flag suggestion seems helpful. I'm able to see that it says the files are not there. I'm guessing I'm not writing the paths correctly. I will tweak that a bit and see if I can fix pathing.
0

First, I want to thank @Bodo and @NationBoneless. I would not have figured this out without their comments. Thank you very much to both of you.

My problem had two parts - one was the problem with concatenation (which I knew from the start) but also in how I was setting up my URL path for curl (which I didn't realize until later).

The correct form for the concatenation is:

curl --ftp-ssl --user myID:mypassword 'ftp://host.mydomain.com' -Q "DELE /public_html/tickets/$i"

Some salient points that I learned along the way and which might be useful to others with similarly limited experience include:

  • Try exactly what the commenters tell you to try, even if you think you understand their reasoning and you think they're on the wrong track - they probably aren't wrong.
  • Use double quotes for shell scripts if you plan to have a variable inside the quotes. Single quotes behave very differently and don't play nice with variables.
  • echo is your friend; it was extremely helpful for debugging because it showed me that my variable was being parsed as $i instead of as the value of $i because of those pesky single quotes. Thanks @Bodo
  • the --verbose flag is very helpful. It let me see that curl was connecting to the server and that my script was failing because it couldn't find the file to delete. My URL path was wrong and I didn't realize it. Thanks @NationBoneless
  • I originally put the full path in the ftp: section and then again after -Q but that doesn't work. After much trial and error I got it to work with just ftp:hostname before the -Q and the rest of the path after the -Q.
  • This next part might not apply to anyone else but the server I'm working on is using WHM/cPanel. The domain to which I'm trying to connect is actually not the domain I am using in my code. I had to connect to host.myserverdomain.com (the domain that I use for root admin stuff in WHM) instead of ftp.thisprojectdomain.com (the domain on which I'm actually working). Apparently WHM/cPanel can be set up many ways and that's how my data center set up mine. Not sure how common this is but if you're tearing your hair out, try switching to the WHM domain.
  • For those wondering why my main script uses both wget and curl; I started with wget because I thought I was more familiar with it. wget can't delete files on the server. I switched to curl for deleting the files but never switched the first statement to curl because it worked. There's probably a way to use curl instead of wget; I leave that exercise to the comments section as I'll stick with what works.
Improve this answer

1 Comment

There is not the correct form, some other variants should work as well. For example 'DELE /public_html/tickets/'"$i" or 'DELE /public_html/tickets/'"${i}" should be equivalent to "DELE /public_html/tickets/$i". In general it is important to use a correct form of quoting depending on what you want. Both double and single quotes have their use cases.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.