Log rotate

At first it was unclear to me why you chose this approach rather than take advantage of the various log rotate options that are available in Traefik (docs).

But it turns out that Traefik does not help with access logs; and you have to manage them yourself.

Usually, I prefer to rely on Linux logrotate for this kind of job. The fact that we are running in Docker does not preclude us from using that kind of mechanism.

Truncating logs

First of all, does the script run as expected? I get this error:

return: can only 'return' from a function or sourced script"

Which makes sense. return should be used to stop the execution of a Bash function. Is the script meant to be called directly?

Probably you mean exit 0, but that is the default exit code anyway. Only when you want to return a non-zero value, should you use exit.

The log file format is in plain text format and not in JSON format, so the log file should still be valid after cleanup

Let's hope so. But I would not be so certain. Some applications may log multi-line text separated by \n, optionally enclosed within quotes. For example the exception stack trace.

Counting lines

Instead of:

lines=$(wc -l "$fn" | awk '{ print $1 }')

You can do it without awk or cut - look at the subtle difference:

lines=$(wc -l < "$fn")

But I find the approach wasteful. Counting lines requires some disk reads and CPU cycles, and this could take seconds on a large file. Instead, we should just rotate the log file once it has reached a certain size in bytes.

I am not fond of truncating the file in-place. Just rotate the whole of it, and start with a new file. Zip the last few log files for reference.

Keeping a backup

A small tip anyway, useful for testing at least:

sed -i.bak "1,${lines_to_del}d" "$fn"

This will keep a copy of the original file, appending a .bak extension so you can compare changes, and recover from errors.

Error handling

At this time, there is no real error handling, other than the set -e command which tells the shell to exit immediately if a command exits with a non-zero status.

I recommend using the trap function in Bash to handle errors or run cleanup routines. This script does one thing that is dangerous: altering a file. On top of that, the file is being used by a critical piece of software.

So it's useful to retain a backup, so that we can recover from any error occurring during this sensitive operation.

Your script should also check that it is touching a file, and not a directory for example. What would happen then?

log rolling races

I can't imagine why you chose to re-invent the wheel here (and the original posting is not so tagged). Log rolling is a mature and well studied need, with good solutions such as the chapter-8 logrotate. With such a solution we would still have lots of historic data to review if we need to compute stats or sleuth the details of some buggy behavior. Long term stats are easier to compute if files hold non-overlapping time ranges of log records.

The fundamental requirement is that a data source, a daemon, shall not consume storage without bound. We model it as producing bytes of log at some rate, and we wish to impose an approximate limit on consumption, subject to inevitable burstiness. This motivates doing log rolling "often". We either keep "lots" of old logs, say 23 stale hourly logs for daily retention, or we often do a cheap stat() like the OP and notice that st_size is still small enough that we silently do nothing and exit with success status. Retaining lots of little files yields much more constant disk usage than keeping, say, just two or three.

We want to "often" do an efficient log roll, so frequent runs will mitigate the size excursions that occasional log activity bursts can produce. The OP's wc -l < ${fn} operation has CPU and I/O cost proportional to file size, and tends to evict other files from the FS cache, reducing the cache's effectiveness. I'm glad to see that we just stat() the file in the "nothing to do" case.

normal behavior leads to error

Doing a return 1 (or exit 1) in that case is very odd, since we successfully determined that there's nothing to do. Also, I don't know how you run this, but the noisy echo for that case is very unusual, since sometimes we may wish to run a log roller from cron, and crond by default will email stdout + stderr to the account we ran under.

race

The rolling operations are

1. sed does an atomic rename() from .log to .log.bak
2. sed reads input, while daemon can still do buffered logging to the .bak file
3. sed re-writes a shorter .log file
4. kill sends a USR1 signal
5. daemon receives the signal, flushes / closes the .bak, re-opens the .log

Any time between sed completing its read() and the daemon completing its close() is the race interval. The daemon can log one or more important messages then, and they will appear only in the (ignored?) .bak file. Roll the file once more and those messages will be gone forever, even if ${lines_to_del} ought to have retained them.

Utilities like logrotate avoid such a race by acting at file granularity, and by patiently waiting to see the daemon complete step (5.) before doing further processing, such as trimming or compressing an old log file.

If you find it convenient to have a "last few hours" file which always has at least a few hundred lines, consider re-writing it as final step of a roller script, with cat ${fn}.1 ${fn}.2 ${fn}.3 > ${fn}.recent. Or use the corresponding date-based filenames.

conversion factor

lines_to_del=$((((size-limit)*lines)/size))
lines_to_del=$((( size-limit)*(lines/size)))

There is of course nothing wrong with the order-of-operations in the OP code. But as a maintainer, I would rather see the 2nd expression, where we're multiplying by unity. It's just easier to reason about when presented as a conversion factor.

Reading the file twice, with wc and sed, is no big deal. But consider caching the \$\frac{lines}{byte}\$ conversion factor in a one-line file, rather than recomputing a figure which largely remains constant.