A Complete Guide to Managing Log Files with Logrotate | Better Stack Community (2024)

Controlling the sizes of log files on a Linux server is crucial due to theircontinuous growth. As log files accumulate, they can consume valuable storagespace, strain server resources, and cause performance and memory issues. Toaddress this problem, log rotation is commonly employed. It involves renaming orcompressing log files before they become too large, while also removing orarchiving old logs to free up storage space. On most Linux distributions, thepreferred tool for log rotation is thelogrotate program, which we will befocusing on in this tutorial.

By reading through this article, you will learn how to:

  • Examine and modify the Logrotate configuration, including both general and application-specific settings.
  • Create Logrotate configurations for a custom application or service.
  • Choose the right log rotation strategy for your application.
  • Debug common log rotation problems.

Prerequisites

Before proceeding with the rest of this tutorial, please ensure that you have:

  • A basic knowledge of working with the Linux command line.
  • A Linux server that includes the non-root user with sudo access. We'll beusing Ubuntu 22.04 throughout in this guide but everything should work even ifyou're on some other distribution.
  • Prior knowledge of how to work with system log files onLinux.
A Complete Guide to Managing Log Files with Logrotate | Better Stack Community (1)
A Complete Guide to Managing Log Files with Logrotate | Better Stack Community (2)

🔭 Want to centralize and monitor your Linux logs?

Head over to Logtail and start ingestingyour logs in 5 minutes.

Why file-based logging matters

Sending your application logs to a file is the first step towards persistingthem and making them available for historical analysis, auditing, andtroubleshooting, although you'll likely want to aggregate them in thecloud to unlock the full potential of your log data.

Even when you've adopted a log management service likeLogtail, we generally don't recommend sendingthe logs to the service directly from the application code for a few reasons:

  1. If a network connection or logging endpoint becomes temporarily unavailable,the application has to attempt resource-intensive retry logic to resume logstreaming, which could impact overall performance.

  2. If there's a persistent outage, the logs could be dropped and lost foreverwhich could impact the troubleshooting process, ability to comply withregulations, or even conduct security investigations.

Therefore, we recommend persisting your logs to a local file to provide someredundancy, then use a log forwarder like Vector totransmit them to their final destination. This approach has a few notableadvantages:

  1. It decouples the log generation process from the log transmission process.This separation of concerns allows the application to focus on its corefunctionality without being concerned about the intricacies of logtransmission. It also simplifies application development and maintenance, asyou can rely on the log forwarder to handle the complexities of log delivery.

  2. Log forwarders can typically aggregate logs from multiple sources, such asdifferent applications or servers, into a centralized location and thisflexibility allows you to adapt your log management infrastructure as yourneeds evolve, without requiring changes to individual applications.

  3. Log forwarders can handle network disruptions, retries, and buffering of logdata so that the log data is delivered reliably even in the event of anextended outage.

  4. They can support different log formats and protocols, making it easier tosend logs to multiple destinations or perform transformations on the logdata.

The necessity of log rotation

Once you've started persisting your logs to local files, you'll need toimplement a process for keeping individual files from becoming too large, andalso a way to remove or archive older logs that are no longer needed to free updisk space.

When log files get too large, they become tedious to work with and searching forthe records relevant to your current tasks can take a long time due to the largevolume of records.

Therefore, implementing log rotation to spread the log data over several filesand to remove older items is a must. It involves renaming log files on apredefined schedule or when the file reaches a predefined size. Once thespecified condition is met, the log file is renamed to preserve its contents andmake way for a new file.

Typically an auto incrementing number or timestamp is appended to the filenameto indicate its time of rotation which is often helpful in narrowing down yoursearch when investigating an issue that occurred on a specific date.

After the file is renamed, a new log file with the same name is created tocapture the latest entries from the application or service. A cleanup process isalso initiated to prevent an accumulation of rotated log files as older logsbeyond a specified retention period are removed. This process repeatsindefinitely as long as the log rotation mechanism is working.

Getting started with Logrotate

The logrotate daemon is pre-installed and active by default in Ubuntu and mostmainstream Linux distributions. If Logrotate is not installed on your machine,ensure to install it first through your distribution's package manager.

Copied!

logrotate --version

Output

logrotate 3.19.0 Default mail command: /usr/bin/mail Default compress command: /bin/gzip Default uncompress command: /bin/gunzip Default compress extension: .gz Default state file path: /var/lib/logrotate/status ACL support: yes SELinux support: yes

The Logrotate daemon uses configuration files to specify all the log rotationdetails for an application. The default setup consists of the following aspects:

  • /etc/logrotate.conf: this is the main configuration file for the Logrotateutility. It defines the global settings and defaults for log rotation that areapplied to all log files unless overridden by individual Logrotateconfiguration files in the /etc/logrotate.d/ directory.
  • /etc/logrotate.d: this directory includes files that configure log rotationpolicies specific to the log files produced by a individual applications orservices.

We will examine both configuration possibilities below.

The main Logrotate configuration

First off, let's view the main Logrotate configuration file at/etc/logrotate.conf. Go ahead and print its contents with the cat utility:

Copied!

cat /etc/logrotate.conf

The command above prints the entire contents of this file:

Copied!

# see "man logrotate" for details# global options do not affect preceding include directives# rotate log files weeklyweekly# use the adm group by default, since this is the owning group# of /var/log/syslog.su root adm# keep 4 weeks worth of backlogsrotate 4# create new (empty) log files after rotating old onescreate# use date as a suffix of the rotated file#dateext# uncomment this if you want your log files compressed#compress# packages drop log rotation information into this directoryinclude /etc/logrotate.d# system-specific logs may also be configured here.

Here's a description of what each of the above configuration directives mean(lines that begin with # indicate a comment):

  • weekly: represents the frequency of log rotation. Alternatively, you canspecify another time interval (hourly, daily, monthly, or yearly).Since the logrotate utility is typical run once per day, you may need tochange this configuration if a if a shorter rotation frequency than daily isdesired (see below).
  • su root adm: log rotation is performed with the root user and admin group.By using this directive, you can ensure that the rotated log files are ownedby a specific user and group, which can be useful for access control andpermissions management. This is particularly relevant when the log files needto be accessed or managed by a specific user or group with appropriateprivileges.
  • rotate 4: log files are rotated four times before old ones are removed. Ifrotate is set to zero, then the old versions are removed immediately and notrotated. If it is set to -1, the older logs will not be remove at all exceptif affected by maxage.
  • create: immediately after rotation, create a new log file with the same nameas the one just rotated.
  • dateext: if this option is enabled, rotated log files will be renamed byappending a date to their filenames, allowing for better organization anddifferentiation of log files based on the date of rotation (especially whenthe frequency of rotation is daily or greater). The default scheme forrotated files is logname.1, logname.2, and so on, but enabling this optionchanges it to logname.YYYYMMDD. You can change the date format through thedateformat and dateyesterday directives.
  • compress: this rule determines whether old log files should be compressed(using gzip by default) or not. Log compression is turned off by default butyou can enable it to save on disk space.
  • include: this directive is used to include additional configuration files orsnippets. It allows you to modularize and organize your Logrotateconfiguration by splitting it into multiple files. In this case, the files inthe /etc/logrotate.d directory have been included in the configuration.

As noted earlier, the /etc/logrotate.conf file serves as a globalconfiguration file for Logrotate, providing default settings and options for logrotation across the system. It sets the stage for log rotation but can beextended or overridden by the configuration files in the /etc/logrotate.d/directory which typically configure the rotation policy for specific applicationlogs.

Application-specific configuration

Next, let's view the contents of the /etc/logrotate.d directory. It typicallycontains additional Logrotate configuration files for various applications orservices installed on your machine:

Copied!

ls /etc/logrotate.d/

Output

alternatives apt btmp rsyslog ufw wtmpapport bootlog dpkg ubuntu-advantage-tools unattended-upgrades

You will observe that quite a few programs have their log rotation configurationin this directory. Each configuration file within /etc/logrotate.d/ focuses ona particular application or log file set, specifying the log file path, rotationfrequency, compression settings, and any additional directives necessary formanaging the logs of that specific application or service.

Having separate configuration files in this directory allows for easycustomization and maintenance of log rotation settings for individualapplications or services without affecting other log files. For example, let'stake a look at the config file for the Rsyslogutility through the cat command:

Copied!

cat /etc/logrotate.d/rsyslog

You'll see the program's output appear on the screen:

Output

/var/log/syslog/var/log/mail.info/var/log/mail.warn/var/log/mail.err/var/log/mail.log/var/log/daemon.log/var/log/kern.log/var/log/auth.log/var/log/user.log/var/log/lpr.log/var/log/cron.log/var/log/debug/var/log/messages{ rotate 4 weekly missingok notifempty compress delaycompress sharedscripts postrotate /usr/lib/rsyslog/rsyslog-rotate endscript}

The above configuration specifies the rotation rules for several log fileslocated in the /var/log/ directory. It also includes the following directivesin addition to the ones we examined in the previous section:

  • missingok: continue log rotation without reporting any error if any of thespecified log files are missing.
  • notifempty: ensures that log files are not rotated if they are empty. If alog file is empty, it won't trigger rotation.
  • delaycompress: delays compression of the rotated log files until the nextrotation cycle. This allows for the previous log file to be available foranalysis before compression.
  • sharedscripts: ensures that the commands or scripts specified in theprerotate or postrotate directive are executed only once, regardless ofthe number of log files being rotated. By default, logrotate executes thecommands/scripts separately for each log file being rotated.
  • postrotate and endscript: encloses the commands or scripts to be executedafter log rotation. In this case, the /usr/lib/rsyslog/rsyslog-rotate scriptis executed after a successful rotation. It sends the SIGHUP signal to theRsyslog service so that it can close and reopen the log file for writing.

Overall, this configuration ensures that the specified log files are rotatedweekly, compressed, and limited to a maximum of 4 rotated log files. It alsoincludes additional directives for handling missing or empty log files andexecutes a post-rotation script specific to Rsyslog.

Other useful directives to note include:

  • size: specifies the maximum size in bytes, kilobytes, megabytes, orgigabytes that a log file can reach before rotation is initiated. This causesthe default schedule to be ignored if as long as size is specified after thetime directive (hourly, daily, etc).

/etc/logrotate.d/myapp

Copied!

/var/log/myapp.log { daily

size 10M

. . .}

In this example, Logrotate will trigger rotation when myapp.log reaches 10megabytes in size. Once the size threshold is crossed, rotation will beinitiated regardless of the time schedule (daily in this case).

  • minsize: the log files are rotated according to the specified time schedule,but not before the specified size is reached. Therefore, when minsize isused, both file size and timestamp are considered to determine if the fileshould be rotated.

/etc/logrotate.d/myapp

Copied!

/var/log/myapp.log { daily

minsize 10M

. . .}

When using minsize, rotation will not occur until the file reaches a minimumof 10 megabytes even if the daily schedule is met.

  • maxsize: specifies that the log files are rotated once they exceed thestated file size, even when the time interval has not yet been reached.

In this snippet, rotation will occur when a size of 10 megabytes is reached.Otherwise, it will rotate weekly.

Choosing the appropriate log rotation strategy

Logrotate offers two directives that specify how the log rotation should behandled: create and copytruncate. The former is the default, and its worksby renaming a log file (say myapp.log) to myapp.log.1, before creating a newmyapp.log file will be created to continue logging.

A Complete Guide to Managing Log Files with Logrotate | Better Stack Community (3)
A Complete Guide to Managing Log Files with Logrotate | Better Stack Community (4)

/etc/logrotate.d/myapp

Copied!

/var/log/myapp.log { rotate 7

create

. . .}

In copytruncate mode, the myapp.log file is copied to a new myapp.log.1file, then the original file is emptied (truncated), allowing the application tocontinue writing to it as if it were a new file. This mode is useful if yourapplication or process does not handle log file rotation gracefully byautomatically switching to the new log file after rotation.

A Complete Guide to Managing Log Files with Logrotate | Better Stack Community (5)
A Complete Guide to Managing Log Files with Logrotate | Better Stack Community (6)

/etc/logrotate.d/myapp

Copied!

/var/log/myapp.log { rotate 7

copytruncate

. . .}

It's worth noting that while copytruncate avoids interrupting the loggingprocess, it may cause a brief period of log loss during the rotation processsince the original file is truncated. However, this is usually acceptable forapplications that don't rely on continuous log analysis and can tolerateoccasional gaps in the logs.

Configuring log rotation for a custom application

So far, we've seen how Logrotate can be used to manage the log files for systemservices and pre-installed utilities on your Linux server. Now, let's look athow to do the same thing for custom applications or services that you'vedeployed to the server.

To simulate an application that writes logs continuously to a file, create thefollowing bash script somewhere on your filesystem. It writes fictional butrealistic-looking log records to a file every second:

logify.sh

Copied!

#!/bin/bashlogfile="/var/log/logify/log_records.log"# Function to generate a random log recordgenerate_log_record() { local loglevel=("INFO" "WARNING" "ERROR") local services=("web" "database" "app" "network") local timestamps=$(date +"%Y-%m-%d %H:%M:%S") local random_level=${loglevel[$RANDOM % ${#loglevel[@]}]} local random_service=${services[$RANDOM % ${#services[@]}]} local message="This is a sample log record for ${random_service} service." echo "${timestamps} [${random_level}] ${message}"}# Main loop to write log records every secondwhile true; do log_record=$(generate_log_record) echo "${log_record}" >> "${logfile}" sleep 1done

Save the file, then make it executable:

Copied!

chmod +x logify.sh

Afterward, create the /var/log/logify directory using elevated privileges,then change the ownership of the directory to your user so that the script canwrite files to the directory:

Copied!

sudo mkdir /var/log/logify

Copied!

sudo chown -R $USER:$USER /var/log/logify/

You can now execute the script and it should begin to write the logs to the fileevery second:

Copied!

./logify.sh

Copied!

cat /var/log/logify/log_records.log

Output

2023-04-29 08:07:25 [WARNING] This is a sample log record for database service.2023-04-29 08:07:26 [ERROR] This is a sample log record for app service.2023-04-29 08:07:27 [INFO] This is a sample log record for app service.2023-04-29 08:07:28 [ERROR] This is a sample log record for app service.2023-04-29 08:07:29 [INFO] This is a sample log record for database service.2023-04-29 08:07:31 [ERROR] This is a sample log record for web service.2023-04-29 08:07:32 [INFO] This is a sample log record for network service.2023-04-29 08:07:33 [INFO] This is a sample log record for network service.. . .

At this stage, you must set up a log rotation policy to prevent thelog_records.log file from growing too large and taking up valuable disk spaceon the server. There are two main options for doing this:

  1. Create a new Logrotate configuration file and place it in the/etc/logrotate.d/ directory to perform log rotation according to thesystem's default schedule (it runs once per day by defaultbut you can change it.

  2. Create the configuration file that is independent of the system's Logrotateschedule and execute Logrotate at your preferred pace using through acronjob.

Creating a standard Logrotate configuration

In this section, you will create a standard configuration file for yourapplication logs and place it in the /etc/logrotate.d/ directory. Go ahead andcreate a new logifyfile in the /etc/logrotate.d/ directory with your texteditor:

Copied!

sudo nano /etc/logrotate.d/logify

Add the following text to the file:

Copied!

/var/log/logify/*.log{ daily missingok rotate 7 compress notifempty}

The configuration above applies to all the files ending with .log in the/var/log/logify/ directory. We've already discussed what each directive heredoes earlier, so we won't go over that again here.

Save the file and test the new configuration by executing the command below. The--debug option instructs logrotate to operate in test mode where only debugmessages are printed.

Copied!

sudo logrotate /etc/logrotate.conf --debug

You should spot an entry for the logify configuration that looks similar towhat is displayed below:

Output

. . .rotating pattern: /var/log/logify/*.log after 1 days (7 rotations)empty log files are not rotated, old logs are removedconsidering log /var/log/logify/log_records.logCreating new state Now: 2023-04-29 10:35 Last rotated at 2023-04-29 10:00 log does not need rotating (log has already been rotated). . .

The above output indicates that the configuration file at/etc/logrotate.d/logify has been found by the logrotate program. Therefore,the log files specified therein will now be rotated according to the definedpolicy along with the other system and application logs.

If you want to test that the log rotation works without without waiting for thespecified schedule, you can use the -f/--force option like this:

Copied!

sudo logrotate -f /etc/logrotate.d/logify

You will observe that the old log file was renamed and compressed and a new onewas created:

Copied!

ls /var/log/logify/

Output

log_records.log log_records.log.1.gz

Another way to verify if a particular log file is rotating or not, and to checkthe last date and time of its rotation, examine the /var/lib/logrotate/statusfile (or /var/lib/logrotate/logrotate.status on Red Hat systems) like this:

Copied!

sudo cat /var/lib/logrotate/status | grep 'logify'

Output

"/var/log/logify/log_records.log" 2023-4-29-10:39:50

Creating a system-independent Logrotate configuration

As mentioned earlier, a system-independent Logrotate configuration is one thatis not run on the default system schedule. Such a configuration will not beincluded in the /etc/logrotate.d/ directory. Instead, you place the file insome other directory and create a cron job that will execute Logrotate with theconfiguration file at custom time interval.

Change into your home directory, and create a logify directory therein:

Copied!

cd ~

Copied!

mkdir logify

Next, edit your logify.sh script and change the logfile variable to thefollowing:

Copied!

logfile="$HOME/logify/log_records.log"

Afterward, rerun the script so that the logs are now written to the ~/logifydirectory:

Copied!

./logify

To create a system-independent Logrotate configuration for these logs, you mustcreate your configuration file outside of /etc/logrotate.d/. Therefore, goahead and create a logrotate.conf file within the ~/logify directory:

Copied!

nano ~/logify/logrotate.conf

Populate the file with the following contents:

logify/logrotate.conf

Copied!

/home/<user>/logify/*.log{ hourly missingok rotate 7 compress notifempty}

This configuration is the same as in the previous section, except that dailyhas been changed to hourly so that the log files are rotated every hourinstead of once per day.

You also need to create a Logrotate state file which stores information such asthe last rotation date and time, the number of rotations performed, and otherrelevant details. This allows Logrotate to accurately perform rotations andprevent unnecessary rotations when they are not required.

In the default Logrotate setup, the state file is located in the/var/lib/logrotate/ directory. However, we will create a custom one throughthe command below:

Copied!

logrotate ~/logify/logrotate.conf --state ~/logify/logrotate.state

The --state option tells logrotate to use an alternative state file locatedat ~/logify/logrotate.state. The logrotate command will create this file ifit doesn't already exist, and you can view its content with cat:

Copied!

cat /home/<user>/logify/logrotate.state

You'll see the program's output appear on the screen:

Output

logrotate state -- version 2"/home/<user>/logify/log_records.log" 2023-4-29-20:0:0

The output indicates that Logrotate identified the relevant log file and when itlast considered them for rotation. The next step here is to set up a cronjob to execute the logrotate file at your desiredfrequency (hourly in this case).

Go ahead and open the cron jobs configuration file by executing crontab -e inyour terminal:

Copied!

crontab -e

The -e option is used to edit the current user's cron jobs using the editorspecified by the $VISUAL or $EDITOR environmental variables. The abovecommand should open a configuration file in your preferred text editor specifiedby one of these variables.

At the bottom of the file, add the following line:

Copied!

0 * * * * /usr/sbin/logrotate /home/<user>/logify/logrotate.conf --state /home/<user>/logify/logrotate.state

This new line specifies that the cron job will be executed every hour (at minute0), and the logrotate command will run with your custom configuration andstate file. The full path of the logrotate binary is used here just to besafe.

Save and close the modified file. You will observe the following output:

Copied!

crontab: installing new crontab

Now that your log rotation policy is all set up, you can view the ~/logifydirectory after an hour to confirm that the log file therein are rotatedaccording to the defined policy. For more details about cron jobs see thefollowing tutorial or type man cron in yourterminal.

Changing the system Logrotate schedule

As mentioned earlier, when using the default system configuration, Logrotateonly runs once per day which means using the hourly option in a configurationwill be ineffective. However, you can modify this behaviour by changing thelocation of the script that runs Logrotate. On Ubuntu, its located at/etc/cron.daily/logrotate which indicates that the script is run once per dayby the system's daily cronjob. If you want to change the schedule to hourly,move the script to the /etc/cron.hourly/ directory using the command below:c

Copied!

sudo mv /etc/cron.daily/logrotate /etc/cron.hourly

Afterward, the script should be executed by the system's hourly cronjob so thatthe hourly option works normally henceforth.

Running commands or scripts before or after log rotation

Logrotate provides the ability to run arbitrary commands or scripts before andafter log rotation through the prerotate and postrotate directives. As theirnames implies, the former executes commands or scripts before log rotationwhile the latter does the same thing after log rotation. Both directives areclosed using the endscript directive.

You can use prerotate to perform any necessary preparations or actionsrequired prior to the rotation, while postrotate should be used to performtasks such as restarting services, notifying stakeholders, or further processingof the rotated log files.

For example, you can monitor your log rotation configuration by pinging amonitoring service like Better Uptime sothat if the rotation does not execute as scheduled, you'll get an alert toinvestigate the problem further.

A Complete Guide to Managing Log Files with Logrotate | Better Stack Community (7)
A Complete Guide to Managing Log Files with Logrotate | Better Stack Community (8)

~/logify/logrotate.conf

Copied!

/home/<user>/logify/*.log{ hourly missingok rotate 7 compress notifempty

sharedscripts

postrotate

curl https://betteruptime.com/api/v1/heartbeat/<heartbeat_id>

endscript

}

In this example, postrotate is used to report that the log rotation wassuccessful according to the defined schedule. If this report is not receivedwithin the expected period, an incident will be created and you will receivenotifications at the configured channels (such as Email, Slack, SMS, etc). Itsalways a good idea to set up such monitoring so that if there's an issue withthe rotation, you catch and fix it quickly before it causes more severeproblems.

Note that postrotate commands or scripts are only executed when at least onefile that matches the specified pattern was rotated. The sharedscriptsdirective above is used to specify that the commands in prerotate andpostrotate blocks should be run only once no matter how many log files wererotated. Normally, the commands are run once per rotated log file which is notideal in this scenario.

A Complete Guide to Managing Log Files with Logrotate | Better Stack Community (9)
A Complete Guide to Managing Log Files with Logrotate | Better Stack Community (10)

If prerotate or postrotate commands or scripts are not executing asexpected, ensure that they have the correct permissions and are executable. Youcan use the chmod +x command to make the scripts or binaries executable whereapplicable. Additionally, double-check that the paths to the scripts areaccurate and that any dependencies required by the scripts are installed.

Modifying Logrotate access permissions

As seen earlier in /etc/logrotate.conf, Logrotate performs its duties with theprivileges of the root user and the adm group. This allows the tool toperform the log rotation operation with elevated permissions, typically requiredto access and manage system logs.

This also means that newly created log files by the tool will be owned by theroot user and group, but this may sometimes prevent the application or serviceproducing the logs from being able to access the file. In such situations, youneed to modify your settings to ensure that the right access permissions are seton the file.

Hence, the create directive provides a few additional options:

/etc/logrotate.d/myapp

Copied!

/var/log/myapp.log{

create 644 <user> <group>

}

In this example, when Logrotate creates a new log file (myapp.log) afterrotation, it will set the file permissions to 644 (read-write for the owner, andread-only for the group and others). The file will be owned by myuser andassigned to mygroup.

Debugging Logrotate problems

You need to ensure that the Logrotate utility is running correctly at all timesso that your scheduled log rotation tasks are executed as expected. If log filesare not rotating as expected, it could be due to incorrect configuration orpermissions issues.

To fix such problems, first check the Logrotate status file at/var/lib/logrotate/status to ensure that the log file is indeed included inthe rotation schedule and to confirm when it was last rotated.

Copied!

sudo cat /var/lib/logrotate/status

If a pattern that matches the log file is not included here, you may need toverify if a corresponding configuration file for the application or service ispresent in the /etc/logrotate.d/ directory.

The logrotate command also provides a helpful -d/--debug option to test anddebug configuration issues by simulating log rotation without actually rotatingthe logs. For example, if you notice that the rotated logs are not beingcompressed and you run logrotate in debug mode, you may observe the followingoutput indicating that the compress directive was misspelled:

Copied!

sudo logrotate /etc/logrotate.d/logify --debug

Output

. . .reading config file /etc/logrotate.d/logifyerror: /etc/logrotate.d/logify:7 unknown option 'compresss' -- ignoring line. . .

Another useful option is -v/--verbose which provides detailed output andinformation about the log rotation process. When enabled, Logrotate displaysadditional messages, including the files being rotated, the actions taken, andany errors or warnings encountered during the rotation.

If you're running logrotate through a cronjob, you can specify the --verboseoption and redirect its standard output and standard error to a file using thesyntax below:

Copied!

0 * * * * /usr/sbin/logrotate -v /home/<user>/logify/logrotate.conf --state /home/<user>/logify/logrotate.state >> </path/to/logrotate.log> 2>&1

For the system cronjob, you must edit the logrotate script that's located in/etc/cron.daily/ by default. Note that when enabling verbose mode here, it'llinclude information about all logs being rotated on the system which can bepretty huge and mostly irrelevant. We recommend using the cron method shownabove if you only care about the logs for a specific service or application.

/etc/cron.daily/logrotate

Copied!

#!/bin/sh# skip in favour of systemd timerif [ -d /run/systemd/system ]; then exit 0fi# this cronjob persists removals (but not purges)if [ ! -x /usr/sbin/logrotate ]; then exit 0fi

/usr/sbin/logrotate -v /etc/logrotate.conf >> </path/to/logrotate.log> 2>&1

EXITVALUE=$?if [ $EXITVALUE != 0 ]; then /usr/bin/logger -t logrotate "ALERT exited abnormally with [$EXITVALUE]"fiexit $EXITVALUE~

The next time Logrotate executes, the logrotate.log file will be created inthe specified directory and you'll find all the details of the log rotation.Here's some example output from a successful rotation attempt:

/path/to/logrotate.log

Copied!

reading config file /home/betterstack/logify/logrotate.confacquired lock on state file /home/betterstack/logify/logrotate.stateReading state from file: /home/betterstack/logify/logrotate.stateAllocating hash table for state file, size 64 entriesCreating new stateCreating new stateCreating new stateCreating new stateHandling 1 logsrotating pattern: /home/betterstack/logify/*.log forced from command line (7 rotations)empty log files are not rotated, old logs are removedconsidering log /home/betterstack/logify/log_records.log Now: 2023-04-30 09:19 Last rotated at 2023-04-30 09:19 log needs rotatingconsidering log /home/betterstack/logify/logrotate.log Now: 2023-04-30 09:19 Last rotated at 2023-04-30 09:00 log does not need rotating (log is empty)rotating log /home/betterstack/logify/log_records.log, log->rotateCount is 7dateext suffix '-2023043009'. . .
A Complete Guide to Managing Log Files with Logrotate | Better Stack Community (11)
A Complete Guide to Managing Log Files with Logrotate | Better Stack Community (12)

Once you're collecting Logrotate logs as above, you can forward them toLogtail so that you can easily search for keyevents and receive alerts when an error is encountered.

A Complete Guide to Managing Log Files with Logrotate | Better Stack Community (13)
A Complete Guide to Managing Log Files with Logrotate | Better Stack Community (14)

Final thoughts

In this tutorial, we explored log rotation in Linux and its implementation usingthe Logrotate program. We began by examining the configuration files anddiscussing key directives commonly encountered. We then created a standardLogrotate configuration for a custom application and then transitioned to asystem-independent configuration, before discussing some common problems withLogrotate and how to troubleshoot them effectively.

To further expand your knowledge of Logrotate and explore its full capabilities,I encourage you to consult its manual page. Simply run man logrotate in yourterminal to access the comprehensive documentation.

Thanks for reading, and happy logging!

A Complete Guide to Managing Log Files with Logrotate | Better Stack Community (15)

Article by

Ayooluwa Isaiah

Ayo is the Head of Content at Better Stack. His passion is simplifying and communicating complex technical ideas effectively. His work was featured on several esteemed publications including LWN.net, Digital Ocean, and CSS-Tricks. When he’s not writing or coding, he loves to travel, bike, and play tennis.

Got an article suggestion?Let us know

Next article

How to View and Manage Systemd Logs with JournalctlLearn how to view and manage systemd logs with journalctl.→

A Complete Guide to Managing Log Files with Logrotate | Better Stack Community (16)

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

A Complete Guide to Managing Log Files with Logrotate | Better Stack Community (2024)
Top Articles
Latest Posts
Article information

Author: Gov. Deandrea McKenzie

Last Updated:

Views: 5648

Rating: 4.6 / 5 (66 voted)

Reviews: 89% of readers found this page helpful

Author information

Name: Gov. Deandrea McKenzie

Birthday: 2001-01-17

Address: Suite 769 2454 Marsha Coves, Debbieton, MS 95002

Phone: +813077629322

Job: Real-Estate Executive

Hobby: Archery, Metal detecting, Kitesurfing, Genealogy, Kitesurfing, Calligraphy, Roller skating

Introduction: My name is Gov. Deandrea McKenzie, I am a spotless, clean, glamorous, sparkling, adventurous, nice, brainy person who loves writing and wants to share my knowledge and understanding with you.