Location>code7788 >text

Linux log management

Popularity：479 ℃/2025-04-18 16:21:19

concept

In Linux systems, log files are records generated during the system operation, which are used to help system administrators monitor system status, troubleshoot problems, detect security events, etc. Linux system log files are usually stored in the /var/log directory, and different log files record different types of information.

Linux system core log files

/var/log/messages

Record various information during the system operation, including hardware device detection, kernel messages, service start and stop status, etc. This is a comprehensive system log file that is commonly used to log system-level events. It contains most of the information generated during the system operation, but does not include certain specific types of information (such as authentication information).

In most Linux distributions (such as Red Hat, CentOS, Fedora, etc.),/var/log/messagesIt is the default system log file.

Example:

root@master-01:~# cat /var/log/messages
Apr 18 15:05:01 master-01 kernel: [5200270.501174][T1101624] overlayfs: NFS export requires "index=on", falling back to nfs_export=off.
Apr 18 15:05:25 master-01 kernel: [5200293.907489][T1058013] overlayfs: NFS export requires "redirect_dir=nofollow" on non-upper mount, falling back to nfs_export=off.
Apr 18 15:05:25 master-01 kernel: [5200293.912834][T1058013] overlayfs: NFS export requires "redirect_dir=nofollow" on non-upper mount, falling back to nfs_export=off.
Apr 18 15:05:25 master-01 kernel: [5200294.053987][T1101321] overlayfs: NFS export requires "index=on", falling back to nfs_export=off.
Apr 18 15:12:49 master-01 kernel: [5200738.730167][T1110236] overlayfs: NFS export requires "index=on", falling back to nfs_export=off.
Apr 18 15:12:50 master-01 kernel: [5200739.651862][T966957] overlayfs: NFS export requires "redirect_dir=nofollow" on non-upper mount, falling back to nfs_export=off.
Apr 18 15:12:50 master-01 kernel: [5200739.655153][T966957] overlayfs: NFS export requires "redirect_dir=nofollow" on non-upper mount, falling back to nfs_export=off.
Apr 18 15:12:50 master-01 kernel: [5200739.665883][T1110256] overlayfs: NFS export requires "index=on", falling back to nfs_export=off.
Apr 18 15:13:12 master-01 kernel: [5200760.882023][T1099873] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready

/var/log/syslog

Record various information during the system operation, including kernel messages, service status, user activities, etc./var/log/syslogThe content usually contains/var/log/messagesMore comprehensively, it not only records system events, but also contains log information for user activities and applications.

In Debian-based systems (such as Ubuntu, Debian, etc.),/var/log/syslogIt is the default system log file.

Example:

root@master-01:~# cat /var/log/syslog
Apr 18 02:42:21 master-01 containerd[29864]: 2025-04-18 02:42:21.544 [INFO][151713]  583: Releasing IP address(es) ContainerID="4760ea90bd55a5807ca183f350d7710e646324735659cb1a8d11b672610604de"
Apr 18 02:42:21 master-01 containerd[29864]: 2025-04-18 02:42:21.544 [INFO][151713]  195: Calico CNI releasing IP address ContainerID="4760ea90bd55a5807ca183f350d7710e646324735659cb1a8d11b672610604de"
Apr 18 02:42:21 master-01 containerd[29864]: 2025-04-18 02:42:21.561 [INFO][151740] ipam_plugin.go 416: Releasing address using handleID ContainerID="4760ea90bd55a5807ca183f350d7710e646324735659cb1a8d11b672610604de" HandleID="k8s-pod-network.4760ea90bd55a5807ca183f350d7710e646324735659cb1a8d11b672610604de" Workload="master--01-k8s-meego--bytedance--bits--river--public--def--577c769ddc--z8vmb-eth0"
Apr 18 02:42:21 master-01 containerd[29864]: time="2025-04-18T02:42:21+08:00" level=info msg="About to acquire host-wide IPAM lock." source="ipam_plugin.go:357"
Apr 18 02:42:21 master-01 containerd[29864]: time="2025-04-18T02:42:21+08:00" level=info msg="Acquired host-wide IPAM lock." source="ipam_plugin.go:372"
Apr 18 02:42:21 master-01 containerd[29864]: 2025-04-18 02:42:21.565 [WARNING][151740] ipam_plugin.go 433: Asked to release address but it doesn't exist. Ignoring ContainerID="4760ea90bd55a5807ca183f350d7710e646324735659cb1a8d11b672610604de" HandleID="k8s-pod-network.4760ea90bd55a5807ca183f350d7710e646324735659cb1a8d11b672610604de" Workload="master--01-k8s-meego--bytedance--bits--river--public--def--577c769ddc--z8vmb-eth0"
Apr 18 02:42:21 master-01 containerd[29864]: 2025-04-18 02:42:21.565 [INFO][151740] ipam_plugin.go 444: Releasing address using workloadID ContainerID="4760ea90bd55a5807ca183f350d7710e646324735659cb1a8d11b672610604de" HandleID="k8s-pod-network.4760ea90bd55a5807ca183f350d7710e646324735659cb1a8d11b672610604de" Workload="master--01-k8s-meego--bytedance--bits--river--public--def--577c769ddc--z8vmb-eth0"
Apr 18 02:42:21 master-01 containerd[29864]: time="2025-04-18T02:42:21+08:00" level=info msg="Released host-wide IPAM lock." source="ipam_plugin.go:378"
Apr 18 02:42:21 master-01 containerd[29864]: 2025-04-18 02:42:21.567 [INFO][151713]  589: Teardown processing complete. ContainerID="4760ea90bd55a5807ca183f350d7710e646324735659cb1a8d11b672610604de"
Apr 18 02:42:21 master-01 containerd[29864]: time="2025-04-18T02:42:21.571116330+08:00" level=warning msg="Failed to get podSandbox status for container event for sandboxID \"4760ea90bd55a5807ca183f350d7710e646324735659cb1a8d11b672610604de\": an error occurred when try to find sandbox: not found. Sending the event with nil podSandboxStatus.

/var/log/(Debian) or/var/log/secure(Red Hat series)

Record information related to user authentication, including user login, password verification, permission changes, etc.

This log file is very important for detecting illegal login attempts and user permission issues. For example, if someone tries to log in to the system with the wrong password, the relevant information will be recorded here.

Example:

root@master-01:~# cat /var/log/
Apr 13 21:55:41 master-01 sshd[4018065]: pam_unix(sshd:auth): check pass; user unknown
Apr 13 21:55:41 master-01 sshd[4018065]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=180.184.100.116
Apr 13 21:55:41 master-01 sshd[4017952]: Connection closed by invalid user bytedance 180.184.100.116 port 55066 [preauth]
Apr 13 21:55:44 master-01 sshd[4018065]: Failed password for invalid user bytedance from 180.184.100.116 port 55436 ssh2
Apr 13 21:55:45 master-01 sshd[4018163]: Invalid user bytedance from 180.184.100.116 port 55632
Apr 13 21:55:45 master-01 sshd[4018163]: pam_unix(sshd:auth): check pass; user unknown
Apr 13 21:55:45 master-01 sshd[4018163]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=180.184.100.116
Apr 13 21:55:45 master-01 sshd[4018065]: Connection closed by invalid user bytedance 180.184.100.116 port 55436 [preauth]
Apr 13 21:55:47 master-01 sshd[4018163]: Failed password for invalid user bytedance from 180.184.100.116 port 55632 ssh2

/var/log/dmesg

Record kernel messages, mainly used to record hardware detection status and kernel running status when system startup.

This file is usually managed directly by the kernel and can be managed by commandsdmesgView its contents. It is very useful for troubleshooting hardware issues and kernel-related errors.

root@master-01:~# dmesg
[5199718.443477] overlayfs: NFS export requires "redirect_dir=nofollow" on non-upper mount, falling back to nfs_export=off.
[5199718.446698] overlayfs: NFS export requires "redirect_dir=nofollow" on non-upper mount, falling back to nfs_export=off.
[5199718.694839] overlayfs: NFS export requires "index=on", falling back to nfs_export=off.
[5199731.304854] overlayfs: NFS export requires "redirect_dir=nofollow" on non-upper mount, falling back to nfs_export=off.
[5199731.344757] overlayfs: NFS export requires "redirect_dir=nofollow" on non-upper mount, falling back to nfs_export=off.
[5199731.717113] overlayfs: NFS export requires "index=on", falling back to nfs_export=off.
[5199762.021051] overlayfs: NFS export requires "redirect_dir=nofollow" on non-upper mount, falling back to nfs_export=off.
[5199762.024317] overlayfs: NFS export requires "redirect_dir=nofollow" on non-upper mount, falling back to nfs_export=off.
[5199762.272861] overlayfs: NFS export requires "index=on", falling back to nfs_export=off.

/var/log/cron

Record the operation status of the cron timing task, including the execution time and execution results of the task.

This file is very useful for monitoring and troubleshooting timing tasks.

Example:

root@master-01:~# cat /var/log/cron
Apr 18 11:00:00 hostname CRON[1234]: (root) CMD (command-to-run)

/var/log/lastlog

Record the information about the last login of the user, including the login time, login terminal, etc.
This file can be passedlastlogCommand to view.

Example:

root@master-01:~# lastlog
Username         Port     From             Latest
root             pts/0    101.126.56.11    Fri Apr 18 15:02:09 +0800 2025
daemon                                     **Never logged in**
bin                                        **Never logged in**
sys                                        **Never logged in**
sync                                       **Never logged in**

View log command

There are many commands to view logs, such as cat, vi, vim, etc., but when the log file is too large, it is not recommended to use cat, vi, vim commands, because these commands will occupy CPU and memory, causing system stuttering.
We should use this timehead、tail、less、moreThese four commands

head

Displays the header information of the file, and the first 10 lines of content are displayed by default.

grammar:

head [Options] File name

Common options description

  • -n: Specify the first N lines of the display file, default 10 lines, this is the most commonly used option
  • -c: Specify the first N characters of the display file

Example:

# The first 10 lines of content are displayed by default
 root@master-01:~# head /var/log/syslog
 Apr 18 00:00:01 master-01 rsyslogd: [origin software="rsyslogd" swVersion="8.1901.0" x-p x-info=""] rsyslogd was HUPed
 Apr 18 00:00:01 master-01 systemd[1]: Started Rotate log files.
 Apr 18 00:00:01 master-01 registry[29766]: time="2025-04-18T00:00:01.857112064+08:00" level=info msg="response completed" =go1.20.8 =":5000" =f940f0ec-f940-4e90-ae08-a3c68bd178db =GET ="10.3.0.28:26990"  ="/v2/larkprivate/.user_public/blobs/sha256:dbda7c5a743e75f8dae27ac9b6adf734d7ea08a8a04445f4c90586ee732b63ab" ="containerd/v1.7.20" ="application/octet-stream" =367.166533ms =200 =252821285
 Apr 18 00:00:01 master-01 registry[29766]: 10.3.0.28 - - [18/Apr/2025:00:00:01 +0800] "GET /v2/larkprivate/.user_public/blobs/sha256:dbda7c5a743e75f8dae27ac9b6adf734d7ea08a8a04445f4c90586ee732b63ab HTTP/1.1" 200 252821285 "" "containerd/v1.7.20"
 Apr 18 00:00:02 master-01 kubelet[31400]: I0418 00:00:02.118970 31400 pod_startup_latency_tracker.go:102] "Observed pod startup duration" pod="default/meego-bytedance-bits-migration-public-def-7c676b59c4-qgd72" podStartSLOduration=-9.223371707735842e+09 ="2025-04-17 23:54:33 +0800 CST"  firstStartedPulling="2025-04-17 23:59:43.921548557 +0800 CST m=+5121628.305810697" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-04-18 00:00:01.947985225 +0800 CST m=+5121646.332247382" watchObservedRunningTime="2025-04-18 00:00:02.118934354  +0800 CST m=+5121646.503196502"
 Apr 18 00:00:05 master-01 kubelet[31400]: E0418 00:00:05.143945 31400 eviction_manager.go:593] "Eviction manager: pod failed to evict" err="timeout waiting to kill pod" pod="default/meego-bytedance-bits-bql-public-def-68c4bc849c-txdjq"
 Apr 18 00:00:05 master-01 kubelet[31400]: I0418 00:00:05.143975 31400 eviction_manager.go:204] "Eviction manager: pods evicted, waiting for pod to be cleaned up" pods="[default/meego-bytedance-bits-bql-public-def-68c4bc849c-txdjq]"
 Apr 18 00:00:06 master-01 registry[29766]: time="2025-04-18T00:00:06.536583529+08:00" level=warning msg="error authorizing context: basic authentication challenge for realm "Registry Realm": invalid authorization credential" =go1.20.8 =":5000" =1ba82cb8-d2f8-46a6-9bc8-867de5a4de67 =GET ="10.3.0.28:54514" ="/v2/"  ="Go-http-client/1.1"
 Apr 18 00:00:06 master-01 registry[29766]: 10.3.0.28 - - [18/Apr/2025:00:00:06 +0800] "GET /v2/ HTTP/1.1" 401 87 "" "Go-http-client/1.1"
 Apr 18 00:00:06 master-01 kernel: [5145977.194403][T3929279] overlayfs: NFS export requires "redirect_dir=nofollow" on non-upper mount, falling back to nfs_export=off.
 root@master-01:~#


 # Specify the first 5 lines of content to display
 root@master-01:~# head -n 5 /var/log/syslog
 Apr 18 00:00:01 master-01 rsyslogd: [origin software="rsyslogd" swVersion="8.1901.0" x-p x-info=""] rsyslogd was HUPed
 Apr 18 00:00:01 master-01 systemd[1]: Started Rotate log files.
 Apr 18 00:00:01 master-01 registry[29766]: time="2025-04-18T00:00:01.857112064+08:00" level=info msg="response completed" =go1.20.8 =":5000" =f940f0ec-f940-4e90-ae08-a3c68bd178db =GET ="10.3.0.28:26990"  ="/v2/larkprivate/.user_public/blobs/sha256:dbda7c5a743e75f8dae27ac9b6adf734d7ea08a8a04445f4c90586ee732b63ab" ="containerd/v1.7.20" ="application/octet-stream" =367.166533ms =200 =252821285
 Apr 18 00:00:01 master-01 registry[29766]: 10.3.0.28 - - [18/Apr/2025:00:00:01 +0800] "GET /v2/larkprivate/.user_public/blobs/sha256:dbda7c5a743e75f8dae27ac9b6adf734d7ea08a8a04445f4c90586ee732b63ab HTTP/1.1" 200 252821285 "" "containerd/v1.7.20"
 Apr 18 00:00:02 master-01 kubelet[31400]: I0418 00:00:02.118970 31400 pod_startup_latency_tracker.go:102] "Observed pod startup duration" pod="default/meego-bytedance-bits-migration-public-def-7c676b59c4-qgd72" podStartSLOduration=-9.223371707735842e+09 ="2025-04-17 23:54:33 +0800 CST"  firstStartedPulling="2025-04-17 23:59:43.921548557 +0800 CST m=+5121628.305810697" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-04-18 00:00:01.947985225 +0800 CST m=+5121646.332247382" watchObservedRunningTime="2025-04-18 00:00:02.118934354  +0800 CST m=+5121646.503196502"

tail

Display the end of the file, the last 10 lines are displayed by default

grammar

tail [Options] File name

Common options description:

  • -n: Specify the last N lines of the display file, default 10 lines
  • -f: Monitor files in real time. When file content is updated, tail will automatically display the added content. The most commonly used
  • -c: Specify the last N characters of the display file

Example:

# The last 10 lines of content are displayed by default
 root@master-01:~# tail /var/log/syslog
 Apr 18 15:41:41 master-01 registry[29766]: time="2025-04-18T15:41:41.29733102+08:00" level=info msg="response completed" =go1.20.8 =":5000" =50602835-b009-4fe0-84d4-5bb7b051ac36 =GET ="10.3.0.13:48422"  ="/v2/dc//blobs/sha256:3d364c24bd4b23073464df3e986c84fbe49dd4ba425702aea2fee938f41be617" ="containerd/v1.7.20" ="application/octet-stream" =81.484271ms =200 =9375912
 Apr 18 15:41:41 master-01 registry[29766]: 10.3.0.13 - - [18/Apr/2025:15:41:41 +0800] "GET /v2/dc//blobs/sha256:3d364c24bd4b23073464df3e986c84fbe49dd4ba425702aea2fee938f41be617 HTTP/1.1" 200 9375912 "" "containerd/v1.7.20"
 Apr 18 15:41:41 master-01 registry[29766]: time="2025-04-18T15:41:41.386849575+08:00" level=info msg="authorized request" =go1.20.8 =":5000" =fcc56214-8346-41d2-9d23-cb12d2ab3e3a =GET ="10.3.0.13:48422"  ="/v2/dc//blobs/sha256:6f8ed6279370767b6e25a753937101e03861f000ba89f13067f49b4f59af1b6d" ="containerd/v1.7.20" ="sha256:6f8ed6279370767b6e25a753937101e03861f000ba89f13067f49b4f59af1b6d" ="dc/"
 Apr 18 15:41:41 master-01 containerd[29864]: time="2025-04-18T15:41:41.699519045+08:00" level=error msg="collecting metrics for 456f3b98e0c6d31ae6e2e95f476689bf326cba5d75edbd8e4b211f9c27c858a9" error="cgroups: cgroup deleted: unknown"
 Apr 18 15:41:42 master-01 registry[29766]: time="2025-04-18T15:41:42.364074963+08:00" level=info msg="response completed" =go1.20.8 =":5000" =fcc56214-8346-41d2-9d23-cb12d2ab3e3a =GET ="10.3.0.13:48422"  ="/v2/dc//blobs/sha256:6f8ed6279370767b6e25a753937101e03861f000ba89f13067f49b4f59af1b6d" ="containerd/v1.7.20" ="application/octet-stream" =1.047214177s =200 =577758195
 Apr 18 15:41:42 master-01 registry[29766]: 10.3.0.13 - - [18/Apr/2025:15:41:41 +0800] "GET /v2/dc//blobs/sha256:6f8ed6279370767b6e25a753937101e03861f000ba89f13067f49b4f59af1b6d HTTP/1.1" 200 577758195 "" "containerd/v1.7.20"
 Apr 18 15:41:50 master-01 kubelet[31400]: I0418 15:41:50.969054 31400 image_gc_manager.go:312] "Disk usage on image filesystem is over the high threshold, trying to free bytes down to the low threshold" usage=85 highThreshold=85 amountToFree=10127867904 lowThreshold=80
 Apr 18 15:41:50 master-01 kubelet[31400]: E0418 15:41:50.971171 31400 :1382] "Image garbage collection failed multiple times in a row" err="Failed to garbage collect required amount of images. Attempted to free 10127867904 bytes, but only found 0 bytes eligible to free."
 Apr 18 15:41:51 master-01 containerd[29864]: time="2025-04-18T15:41:51.718029829+08:00" level=error msg="collecting metrics for 456f3b98e0c6d31ae6e2e95f476689bf326cba5d75edbd8e4b211f9c27c858a9" error="cgroups: cgroup deleted: unknown"
 Apr 18 15:42:01 master-01 containerd[29864]: time="2025-04-18T15:42:01.740866578+08:00" level=error msg="collecting metrics for 456f3b98e0c6d31ae6e2e95f476689bf326cba5d75edbd8e4b211f9c27c858a9" error="cgroups: cgroup deleted: unknown"

 # Specify the content of the last 5 lines displayed
 root@master-01:~# tail -n 5 /var/log/syslog
 Apr 18 15:42:06 master-01 registry[29766]: 10.3.0.13 - - [18/Apr/2025:15:42:06 +0800] "GET /v2/larkprivate/.user_public/blobs/sha256:93d5206170400f57c1f3b57c56beb22ebab609af44bd7335e6880962cfd5e125 HTTP/1.1" 200 2137 "" "containerd/v1.7.20"
 Apr 18 15:42:06 master-01 registry[29766]: time="2025-04-18T15:42:06.333661489+08:00" level=info msg="authorized request" =go1.20.8 =":5000" =5891580a-2d84-435c-a908-cb296e199b7b =GET ="10.3.0.13:43656"  ="/v2/larkprivate/.user_public/blobs/sha256:43d77a3ecc89b1cb6cebdd49083c63dfc24faac7309a55d2c8fd91dbd5d7fa4a" ="containerd/v1.7.20" ="sha256:43d77a3ecc89b1cb6cebdd49083c63dfc24faac7309a55d2c8fd91dbd5d7fa4a" ="larkprivate/.user_public"
 Apr 18 15:42:06 master-01 registry[29766]: time="2025-04-18T15:42:06.343030264+08:00" level=info msg="response completed" =go1.20.8 =":5000" =5891580a-2d84-435c-a908-cb296e199b7b =GET ="10.3.0.13:43656"  ="/v2/larkprivate/.user_public/blobs/sha256:43d77a3ecc89b1cb6cebdd49083c63dfc24faac7309a55d2c8fd91dbd5d7fa4a" ="containerd/v1.7.20" ="application/octet-stream" =79.554385ms =200 =9374304
 Apr 18 15:42:06 master-01 registry[29766]: 10.3.0.13 - - [18/Apr/2025:15:42:06 +0800] "GET /v2/larkprivate/.user_public/blobs/sha256:43d77a3ecc89b1cb6cebdd49083c63dfc24faac7309a55d2c8fd91dbd5d7fa4a HTTP/1.1" 200 9374304 "" "containerd/v1.7.20"
 Apr 18 15:42:11 master-01 containerd[29864]: time="2025-04-18T15:42:11.760033888+08:00" level=error msg="collecting metrics for 456f3b98e0c6d31ae6e2e95f476689bf326cba5d75edbd8e4b211f9c27c858a9" error="cgroups: cgroup deleted: unknown"

 # Real-time monitoring
 root@master-01:~# tail -f /var/log/syslog
 Apr 18 15:42:05 master-01 registry[29766]: 10.3.0.13 - - [18/Apr/2025:15:42:05 +0800] "GET /v2/larkprivate/.user_public/blobs/sha256:dbda7c5a743e75f8dae27ac9b6adf734d7ea08a8a04445f4c90586ee732b63ab HTTP/1.1" 200 252821285 "" "containerd/v1.7.20"
 Apr 18 15:42:06 master-01 registry[29766]: time="2025-04-18T15:42:06.044094175+08:00" level=info msg="response completed" =go1.20.8 =":5000" =6a20742d-0b29-4aa9-a081-622579e34913 =GET ="10.3.0.13:43656"  ="/v2/larkprivate/.user_public/blobs/sha256:3d82803473ffa929bd62ccff81e5f9695fd1dfe883dcaecad2d91c350a51f1a1" ="containerd/v1.7.20" ="application/octet-stream" =269.30401ms =200 =87241090
 Apr 18 15:42:06 master-01 registry[29766]: 10.3.0.13 - - [18/Apr/2025:15:42:05 +0800] "GET /v2/larkprivate/.user_public/blobs/sha256:3d82803473ffa929bd62ccff81e5f9695fd1dfe883dcaecad2d91c350a51f1a1 HTTP/1.1" 200 87241090 "" "containerd/v1.7.20"

less

Display file contents by page, and will automatically jump to the first page when the last page is reached.

Operation method:

Next page: Space or CTRL+F (front)
     Previous page: CTRL+B (back)
     Search:/Search content
     First line: g
     Last line: G
     Line 100: 100g or enter 100 to enter
     Exit: Press q

Example:

root@master-01:~# less /var/log/syslog

more

Display file contents by page and exit on the last page

Operation method:

Next page: Space or CTRL+F (front)
     Previous page: CTRL+B (back)
     Search:/Search content
     First line: g
     Last line: G
     Line 100: 100g or enter 100 to enter
     Exit: Press q

Example:

root@master-01:~# more /var/log/syslog

Log Cutting

In Linux systems, log cutting (Log Rotation) is an important part of log management, which is used to prevent unlimited growth of log files, save disk space, and facilitate log management and analysis. Here are several common log cutting methods:

logrotate tool

logrotateIt is a Linux system log management tool. You can cut and compress a single log file or a file in a certain directory by time/size; specify the number of logs saved; and you can also run custom commands after cutting.

logrotateThe main function is to regularly check the log files specified in the configuration file and perform the following operations on the log files according to configuration conditions (such as file size, date, etc.):

  • Rotate: Rename the current log file to a new file (usually the date or sequence number).
  • Create a new log file: Recreate an empty log file so that the program can continue to write.
  • Compression: Compress old log files to save disk space.
  • Delete old logs: Delete expired log files according to configuration.
  • Execute script: Execute custom scripts before and after rotation, such as reloading the service.
  • logrotate is usually triggered by the cron timing task and runs once a day.
  • Configuration instance

logrotateThe main configuration file is/etc/,at the same time/etc//Log rotation configuration files for specific applications can be stored in the directory.logrotateThe main configuration file will be read first, then the/etc//Configuration file in the directory.

System default configuration file

Take the Debian system as an example:

View the main configuration file

root@master-01:~# cat /etc/
# see "man logrotate" for details
# rotate log files weekly
weekly
# keep 4 weeks worth of backlogs
rotate 4
# create new (empty) log files after rotating old ones
create
# use date as a suffix of the rotated file
#dateext
# uncomment this if you want your log files compressed
#compress
# packages drop log rotation information into this directory
include /etc/

Check/etc/Sub-configuration files in the directory

root@master-01:~# cat /etc//rsyslog
/var/log/syslog
{
        rotate 7
        daily
        missingok
        notifempty
        delaycompress
        compress
        postrotate
                /usr/lib/rsyslog/rsyslog-rotate
        endscript
}

/var/log/
/var/log/
/var/log/
/var/log/
/var/log/
/var/log/
/var/log/
/var/log/
/var/log/
/var/log/
/var/log/debug
/var/log/messages
{
        rotate 4
        weekly
        missingok
        notifempty
        compress
        delaycompress
        sharedscripts
        postrotate
                /usr/lib/rsyslog/rsyslog-rotate
        endscript
}

Detailed explanation of the configuration file

# Log file name, you can fill in multiple
 /var/log/syslog
 {
         # Specify how many old log files are retained, here we specify how many old log files are retained
         rotate 7
         # Specify how long it takes to rotate. Here we specify rotations every day. Weekly specifies rotations once a week. Monthly indicates rotations once a month.
         daily
         # If the log file is lost, no error will be reported
         Missingok
         # If the log file is empty, no rotation will occur
         notifempty
         # Delay compressing log files until the next rotation, which can avoid immediate compression during rotation and reduce disk I/O
         delaycompress
         # Compress old log files
         Compress
         # Permissions, owners, and groups when creating new log files
         create 640 root adm
         # Specify the script that is executed after rotation, for example, reloading the service or sending notifications.  Use in conjunction with endscript
         postrotate
                 /usr/lib/rsyslog/rsyslog-rotate
         # script end identifier, used in conjunction with postrotate
         endscript
 }

Common configurations are as follows:

rotate：Specify the number of old log files to be retained。  For example, rotate 4 means keeping 4 old log files.
 daily/weekly/monthly：Specify the rotation frequency。  daily means rotation once a day, weekly means rotation once a week, monthly means rotation once a month.
 size: Rotate according to the size of the log file.  For example, size 100k means rotation when the file size exceeds 100KB.
 compress：Compress old log files。
 delaycompress: Delay compression until the next rotation.  This avoids immediate compression during rotation and reduces disk I/O.
 Missingok: If the log file is lost, there will be no error.
 notifempty: If the log file is empty, it will not rotate.
 create: Specifies the permissions, owner, and group of the new log file. For example，create 640 root adm。
 sharedscripts: All log file sharing scripts (such as postrotate).
 postrotate and endscript: scripts executed after rotation. For example，Reload the service or send notifications。

How to automatically run logrotate

logrotateUsually bycronThe timing tasks run automatically. exist/etc//、/etc//or/etc//There will be a directorylogrotateScripts, responsible for regular callslogrotateOrder. We just need to write itlogrotateJust configure the file.

Example:

root@master-01:~# cat /etc//logrotate
#!/bin/sh

# skip in favour of systemd timer
if [ -d /run/systemd/system ]; then
    exit 0
fi

# this cronjob persists removals (but not purges)
if [ ! -x /usr/sbin/logrotate ]; then
    exit 0
fi

/usr/sbin/logrotate /etc/
EXITVALUE=$?
if [ $EXITVALUE != 0 ]; then
    /usr/bin/logger -t logrotate "ALERT exited abnormally with [$EXITVALUE]"
fi
exit $EXITVALUE

How to run logrotate manually?

  • Debug mode:

Before enforcing logrotate, it is recommended to use debug mode to check if the configuration file is correct.The debug mode does not actually perform rotation operations, but the operation to be performed will be displayed.. This can avoid problems caused by configuration errors.

# grammar
 logrotate -d [logrotate configuration file]

 # Debug all log files
 logrotate -d /etc/

 # Debug a single log file
 logrotate -d /etc//apt
  • Force rotation log file

Forced rotation may cause log files to be rotated repeatedly, or in some cases overwrite old log files

# grammar
 logrotate -f [logrotate configuration file]

 # Force rotation of all log files
 logrotate -f /etc/

 # Force rotation of single log files
 logrotate -f /etc//apt

splittool

splitThe command is used to split a large file into multiple small files, and supports splitting by number of lines, file size or specified number.

Basic syntax

split [Options] [Input File] [Output File Prefix]

Common options:

  • -l: Split the file by the specified number of lines
# This will split into multiple files, each containing 100 lines, and the output file name is outputfileaa, outputfileab, etc.
 split -l 100 outputfile
  • -b: Split files according to the specified size
# This will split into multiple files, each file size is about 10MB
 split -b 10M outputfile
  • -n: Split files by specified number
# This will split into 5 files
 split -n 5 outputfile
  • -a: Specify the length of the generated file suffix
# This will generate files with 3-bit suffixes, such as outputfileaaa, outputfileaab
 split -l 100 -a 3 outputfile
  • -d: Use numeric suffix instead of letters
# This will generate files with the file suffixed by numbers, such as outputfile00, outputfile01
 split -l 100 -d outputfile
<Previous Next>