Linux

Influxdb examples: Enhancing Your Time-Series Data Management

Leave a comment

Introduction

InfluxDB, a widely-used open-source time series database, excels in handling large volumes of time-stamped data for applications like monitoring systems, IoT devices, and financial tracking. This tutorial will guide you through querying InfluxDB, demonstrating practical examples and setup instructions.

If you haven’t installed InfluxDB yet, refer to the installation guide provided earlier to get started. This introduction sets the stage for you to effectively manage and analyze time-series data using InfluxDB’s powerful features.

InfluxDB examples

InfluxDB show databases

[root@MonitoringServer ~]# influx
Connected to http://localhost:8086 version 1.7.4
InfluxDB shell version: 1.7.4
Enter an InfluxQL query

> show databases                                                                                                         
name: databases
name
----
_internal
devopsrolesDB
telegraf

Use databases

> use devopsrolesDB                                                                                                              
Using database devopsrolesDB
>

Uptime Server

> select last("uptime_format") as "value" from "system" where "host" =~ /DevopsRoles\.com$/ AND time >= now() - 1h GROUP BY time(60s)

Check Root FS used

> SELECT last("used_percent") FROM "disk" WHERE ("host" =~ /^DevopsRoles\.com$/ AND "path" = '/') AND time >= now() -6h GROUP BY time(5m) fill(null)

Swap used

> SELECT last("used_percent") FROM "swap" WHERE ("host" =~ /^DevopsRoles\.com$/) AND time >= now() -1h GROUP BY time(5m) fill(null)

Users login

> SELECT last("n_users") FROM "system" WHERE ("host" =~ /^DevopsRoles\.com$/) AND time >= now() -1h GROUP BY time(5m) fill(null)

CPU usage

> SELECT last("usage_idle") * -1 + 100 FROM "cpu" WHERE ("host" =~ /^DevopsRoles\.com$/ AND "cpu" = 'cpu-total') AND time >= now() -1h GROUP BY time(5m) fill(null)

RAM Usage

> SELECT last("used_percent") FROM "mem" WHERE ("host" =~ /^DevopsRoles\.com$/) AND time >= now() -1h GROUP BY time(5m) fill(null)

CPU Load

> SELECT mean(load1) as load1,mean(load5) as load5,mean(load15) as load15  FROM "system" WHERE host =~ /^DevopsRoles\.com$/ AND time >= now() -1h GROUP BY time(5m) fill(null)

CPUs number

>  SELECT last("n_cpus") FROM "system" WHERE ("host" =~ /^DevopsRoles\.com$/) AND time >= now() -1h GROUP BY time(5m) fill(null)

Other Influxdb examples

How to list all value systems, swap, CPUs, Memory, and so on.

Enter as following for the system

> select * from "system" where host =~ /^DevopsRoles\.com$/ AND time >= now() -1h

## The output as below:
name: system
time                host                         load1 load15 load5 n_cpus n_users uptime  uptime_format
----                ----                         ----- ------ ----- ------ ------- ------  -------------
1574665340000000000 DevopsRoles.com 0.27  0.03   0.11  4      1       8105215 93 days, 19:26
1574665350000000000 DevopsRoles.com 0.22  0.03   0.1   4      1       8105225 93 days, 19:27
1574665360000000000 DevopsRoles.com 0.19  0.03   0.1   4      1       8105235 93 days, 19:27

CPU

> select * from "cpu" where host =~ /^DevopsRoles\.com$/ AND time >= now() - 120s

## The output as below:                                                                                                                                             
name: cpu
time                cpu       host                         usage_guest usage_guest_nice usage_idle        usage_iowait        usage_irq usage_nice usage_softirq        usage_steal          usage_system         usage_user
----                ---       ----                         ----------- ---------------- ----------        ------------        --------- ---------- -------------        -----------          ------------         ----------
1574670090000000000 cpu-total DevopsRoles.com 0           0                99.92494371410935 0                   0         0          0                    0                    0.025018764076678877 0.050037528153357755
1574670090000000000 cpu0      DevopsRoles.com 0           0                100               0                   0         0          0                    0                    0                    0
1574670090000000000 cpu1      DevopsRoles.com 0           0                99.89989990213955 0                   0         0          0                    0                    0.1001001000954934   0
1574670090000000000 cpu2      DevopsRoles.com 0           0                99.89979960143319 0                   0         0          0                    0                    0.10020040080409609  0
1574670090000000000 cpu3      DevopsRoles.com 0           0                100               0                   0         0          0                    0                    0                    0
1574670100000000000 cpu-total DevopsRoles.com 0           0                99.79989994515057 0.12506253122346286 0         0          0                    0                    0.05002501250212444  0.02501250625561197
1574670100000000000 cpu0      DevopsRoles.com 0           0                99.49949949205266 0.5005005005184352  0         0          0                    0                    0                    0
1574670100000000000 cpu1      DevopsRoles.com 0           0                100               0                   0         0          0                    0                    0                    0
1574670100000000000 cpu2      DevopsRoles.com 0           0                99.79999999517575 0                   0         0          0                    0                    0.09999999999286956  0.09999999998377461
1574670100000000000 cpu3      DevopsRoles.com 0           0                100               0                   0         0          0                    0                    0                    0
1574670110000000000 cpu-total DevopsRoles.com 0           0                99.64982491096929 0.22511255633968244 0         0          0                    0.025012506253392856 0.05002501250223596  0.05002501249768622

DISK

> select * from "disk" where host =~ /^DevopsRoles\.com$/ AND time >= now() - 120s

## The output as below:                                                                                                                                            
name: disk
time                device                         free        fstype host                         inodes_free inodes_total inodes_used mode path         total        used        used_percent
----                ------                         ----        ------ ----                         ----------- ------------ ----------- ---- ----         -----        ----        ------------
1574670150000000000 10.10.10.225:/mnt_nfs/data_volume/ 64795705344 nfs4   DevopsRoles.com 5924993     6553600      628607      rw   /mnt_nfs/data 105554903040 35373711360 35.313883742109724
1574670150000000000 mapper/VolGroup-lv_root        40046198784 ext4   DevopsRoles.com 3014316     3182400      168084      rw   /            51484815360  8823488512  18.0551360162319
1574670150000000000 vda1                           427900928   ext4   DevopsRoles.com 127976      128016       40          rw   /boot        507744256    53628928    11.137196859502726
1574670150000000000 vdb1                           9870200832  ext4   DevopsRoles.com 655325      655360       35          rw   /app         10568843264  161775616   1.612599639149392

Diskio

> select * from "diskio" where host =~ /^DevopsRoles\.com$/ AND time >= now() - 120s

## The output as below:              
name: diskio
time                host                         io_time   iops_in_progress name read_bytes read_time reads  weighted_io_time write_bytes write_time writes
----                ----                         -------   ---------------- ---- ---------- --------- -----  ---------------- ----------- ---------- ------
1574670240000000000 DevopsRoles.com 137167292 0                dm-0 2659918848 2451413   181804 3747940429       51691593728 3745384372 12620365
1574670240000000000 DevopsRoles.com 3102      0                vdb1 5949440    3049      770    5493             315904      2445       59
1574670240000000000 DevopsRoles.com 6376      0                dm-1 7897088    31096     1928   68191            27774976    37096      6781
1574670240000000000 DevopsRoles.com 137161235 0                vda  2674118656 1651142   137682 1886179382       51719428096 1884529301 4253428
1574670240000000000 DevopsRoles.com 667       0                vda1 2124800    370       521    667              47104       297        19
1574670240000000000 DevopsRoles.com 107       0                sr0  155648     107       49     107              0           0          0

Kernel

> select * from "kernel" where host =~ /^DevopsRoles\.com$/ AND time >= now() - 120s

## The output as below:                                                                                                                                           
name: kernel
time                boot_time  context_switches entropy_avail host                         interrupts processes_forked
----                ---------  ---------------- ------------- ----                         ---------- ----------------
1574670390000000000 1566560125 897266212        1320          DevopsRoles.com 611974729  543719
1574670400000000000 1566560125 897267347        1320          DevopsRoles.com 611975497  543719
1574670410000000000 1566560125 897268311        1320          DevopsRoles.com 611976101  543719
1574670420000000000 1566560125 897269308        1355          DevopsRoles.com 611976734  543719
1574670430000000000 1566560125 897270363        1396          DevopsRoles.com 611977420  543719
1574670440000000000 1566560125 897271391        1412          DevopsRoles.com 611978084  543719
1574670450000000000 1566560125 897272328        1412          DevopsRoles.com 611978685  543719
1574670460000000000 1566560125 897273390        1423          DevopsRoles.com 611979457  543719
>

Network

> select bytes_recv,bytes_sent,drop_in,drop_out from "net" where host =~ /^DevopsRoles\.com$/ AND time >= now() - 120s

## The output as below:                                                                                                        
name: net
time                bytes_recv  bytes_sent  drop_in drop_out
----                ----------  ----------  ------- --------
1574670830000000000 42310540034 60204453178 0       0
1574670840000000000 42310549919 60204469772 0       0
1574670850000000000 42310565133 60204488497 0       0
1574670860000000000 42310577265 60204503755 0       0
1574670870000000000 42310587249 60204520594 0       0
1574670880000000000 42310613504 60204538330 0       0

Processes

> select * from "processes" where host =~ /^DevopsRoles\.com$/ AND time >= now() - 120s

## The output as below:                                                                                                                                         
name: processes
time                blocked dead host                         idle paging running sleeping stopped total total_threads unknown zombies
----                ------- ---- ----                         ---- ------ ------- -------- ------- ----- ------------- ------- -------
1574670990000000000 0       0    DevopsRoles.com 0    0      0       126      0       126   209           0       0
1574671000000000000 0       0    DevopsRoles.com 0    0      0       126      0       126   209           0       0
1574671010000000000 0       0    DevopsRoles.com 0    0      0       126      0       126   210           0       0
1574671020000000000 0       0    DevopsRoles.com 0    0      0       126      0       126   210           0       0
1574671030000000000 0       0    DevopsRoles.com 0    0      0       126      0       126   210           0       0
1574671040000000000 0       0    DevopsRoles.com 0    0      0       126      0       126   210           0       0
1574671050000000000 0       0    DevopsRoles.com 0    0      0       126      0       126   210           0       0
1574671060000000000 0       0    DevopsRoles.com 0    0      0       126      0       126   210           0       0

swap

> select * from "swap" where host =~ /^DevopsRoles\.com$/ AND time >= now() - 120s

## The output as below:                                                                                                                                              
name: swap
time                free      host                         in      out      total     used     used_percent
----                ----      ----                         --      ---      -----     ----     ------------
1574671030000000000 831287296 DevopsRoles.com 6680576 27774976 855629824 24342528 2.8449835801889956
1574671040000000000 831287296 DevopsRoles.com 6680576 27774976 855629824 24342528 2.8449835801889956
1574671050000000000 831287296 DevopsRoles.com 6680576 27774976 855629824 24342528 2.8449835801889956
1574671060000000000 831287296 DevopsRoles.com 6680576 27774976 855629824 24342528 2.8449835801889956
1574671070000000000 831287296 DevopsRoles.com 6680576 27774976 855629824 24342528 2.8449835801889956
1574671080000000000 831287296 DevopsRoles.com 6680576 27774976 855629824 24342528 2.8449835801889956
1574671090000000000 831287296 DevopsRoles.com 6680576 27774976 855629824 24342528 2.8449835801889956
1574671100000000000 831287296 DevopsRoles.com 6680576 27774976 855629824 24342528 2.8449835801889956
1574671110000000000 831287296 DevopsRoles.com 6680576 27774976 855629824 24342528 2.8449835801889956

How to show tag values.

SHOW TAG VALUES FROM system WITH KEY=host
SHOW TAG VALUES FROM "cpu" WITH KEY = "cpu" WHERE host =~ /$server/
SHOW TAG VALUES FROM "disk" WITH KEY = "device"
SHOW TAG VALUES FROM "net" WITH KEY = "interface" WHERE host =~ /$server/

Conclusion

Through the article, How to query Influxdb examples above. InfluxDB is widely used in various domains, including DevOps, IoT, monitoring and observability, and real-time analytics, due to its high performance, scalability, and ease of use. I hope will this your helpful. Thank you for reading DevOpsRoles.com page

Linux

Linux understand Page cache and buffer cache

Leave a comment

In this tutorial, I have written about Linux understand Page cache and buffer cache in Linux System.

Most file-system cache data read from disk.

Linux understand Page cache

What does Page cache work?

A cache of data is accessed via the file system.

How to check page cache is actually used.

Create a large file

[root@DevopsRoles ~]# mkdir /test
[root@DevopsRoles ~]# dd if=/dev/zero of=/test/large.txt count=100 bs=10M
100+0 records in
100+0 records out
1048576000 bytes (1.0 GB) copied, 1.62731 s, 644 MB/s
[root@DevopsRoles ~]# echo 3 > /proc/sys/vm/drop_caches

Check memory usage before putting it in the page cache

[root@DevopsRoles ~]# vmstat
 procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
  r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
  1  0      0 377344      0  64336    0    0   441  5203  131  293  0  2 97  0  0

[root@DevopsRoles ~]# cat /test/large.txt > /dev/null

Check memory usage after getting on page cache

[root@DevopsRoles ~]# vmstat 
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 2  0      0   6500      0 435008    0    0  3265  3008  108  179  0  2 98  0  0

We will confirm that accessing data in the cache is fast.

Run command 1s

[root@DevopsRoles ~]# time cat /test/large.txt > /dev/null

real	0m1.068s
user	0m0.003s
sys	0m0.987s

Run command 2s

[root@DevopsRoles ~]# time cat /test/large.txt > /dev/null

real	0m1.064s
user	0m0.003s
sys	0m0.981s

Linux understand buffer cache

What does Buffer cache work?

Cache data accessed via raw I/O. It is a page cache for block devices.

How to check Buffer cache is actually used.

[root@DevopsRoles ~]# vmstat
 procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
  r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
  1  0      0   5668      0 435832    0    0  6434  2087  116  132  0  2 98  0  0
 [root@DevopsRoles ~]# dd if=/dev/sda of=/dev/null count=100 bs=10M
 100+0 records in
 100+0 records out
 1048576000 bytes (1.0 GB) copied, 1.59043 s, 659 MB/s

Increase buffer cache (buff)

We will confirm that accessing data in the cache is fast.

Run command 1s

[root@DevopsRoles ~]# vmstat
 procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
  r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
  2  0      0  13408 359528  68520    0    0  7715  1888  123  127  0  2 98  0  0
 [root@DevopsRoles ~]# time dd if=/dev/sda of=/dev/null count=100 bs=10M
 100+0 records in
 100+0 records out
 1048576000 bytes (1.0 GB) copied, 1.13208 s, 926 MB/s
 real    0m1.138s
 user    0m0.001s
 sys    0m1.068s

Run command 2s

[root@DevopsRoles ~]# vmstat
 procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
  r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
  2  0      0  13664 359220  68520    0    0  8896  1747  130  119  0  2 98  0  0
 [root@DevopsRoles ~]# time dd if=/dev/sda of=/dev/null count=100 bs=10M
 100+0 records in
 100+0 records out
 1048576000 bytes (1.0 GB) copied, 1.13821 s, 921 MB/s
 real    0m1.144s
 user    0m0.001s
 sys    0m1.072s

Conclusion

Linux understand Page cache and buffer cache. I hope will this your helpful. Thank you for reading the DevopsRoles page!

Jenkins

Mastering Jenkins pipeline groovy example

Leave a comment

Introduction

In this tutorial, I have written a script groovy using Jenkins pipeline call shell to create a folder and copy. How to Execute shell script from Jenkins groovy script in Pipeline. Now, let’s go to the Jenkins pipeline groovy example.

Jenkins Pipeline creates multiple automation jobs with the help of use cases and runs them as a Jenkins pipeline.

You can installed build pipeline plugin on Jenkins server.

Jenkins pipeline groovy example

I will create 3 folder: app1,app-api,app2 and copy war file is app1.war,app-api.war,app2.war

JENKINS_HOME: /var/lib/jenkins
WORKSPACE: /var/lib/jenkins/{JOB_NAME}

Execute shell script from Jenkins Groovy script in Pipeline.

node('master') {
stage('Create directory and copy to folder release') {
   artifacts = "app1,app-api,app2"
   targets = artifacts.split(",")
   for (String artifact : targets){
         Warfile = artifact + ".war"
         sh """
              mkdir -p ${JENKINS_HOME}/delivery/${artifact}           
              cp ${WORKSPACE}/${artifact}/target/${Warfile} ${JENKINS_HOME}/delivery/${artifact}/
         """
 }
 }
}

Conclusion

Throughout this article, “How to Execute a Shell Script from Jenkins Groovy Script in Pipeline,” we’ve explored detailed steps and strategies for integrating shell scripts into Jenkins pipelines using Groovy. I hope you found the information provided useful for enhancing your DevOps processes. Thank you for reading at DevOpsRoles.com, and stay tuned for more insights and tutorials to streamline your development and operational tasks.

Database Administrator

Setting Up Oracle Automatic Startup on Linux: A Comprehensive Guide

Leave a comment

Introduction

In this tutorial, How to confirm Oracle automatic startup on Linux. How to make Oracle start automatically in Linux.

Ensuring that your Oracle database starts automatically when your Linux system boots up can save time and reduce manual intervention, enhancing the reliability of your database operations.

This guide will walk you through the process of configuring automatic startup for Oracle on a Linux system. By following these steps, you can ensure that your Oracle database is always ready to handle your data needs, even after a system reboot.

By default, Oracle software installation does not deploy automatic startup and shutdown init scripts on the platform.

How to confirm Oracle automatic startup on Linux.

The dbstart utility reads the oratab file. Confirm it in the example below

[HuuPV@DevopsRoles ~]$ sudo su - oracle
[oracle@DevopsRoles ~]$ cat /etc/oratab

 DEVOPSROLES_SID:/opt/oracle/product/11.2.0/dbhome_1:Y 
 DEVOPSROLES_SID02:/opt/oracle/product/10.2.03/dbhome_2:N

We see there are two instances on this server. Oracle 10.2.03 is marked “N” and will not restart when the Linux OS reboots. Oracle 11.2.0 is marked “Y” and will restart when the Linux OS reboots.

Auto Start Oracle on Linux

1. In the /etc/oratab file with the autostart column to “Y”

[oracle@DevopsRoles ~]$ cat /etc/oratab
DEVOPSROLES_SID:/opt/oracle/product/11.2.0/dbhome_1:Y

2. Create the file named “oracle” in /etc/init.d folder.

[root@DevopsRoles ~]# cd /etc/init.d
[root@DevopsRoles init.d]# vi oracle

#!/bin/sh
ORACLE_HOME=/opt/oracle/product/11.2.0/dbhome_1
ORACLE_OWNER=oracle
case "$1" in
'start') # Start the Oracle databases and listeners
su - $ORACLE_OWNER -c "$ORACLE_HOME/bin/dbstart $ORACLE_HOME"
;;
'stop') # Stop the Oracle databases and listeners
su - $ORACLE_OWNER -c "$ORACLE_HOME/bin/dbshut $ORACLE_HOME"
;;
esac

3. Create a symbolic link

[root@DevopsRoles ~]# ln -s /etc/init.d/oracle /etc/rc0.d/K10oracle
[root@DevopsRoles ~]# ln -s /etc/init.d/oracle /etc/rc3.d/S99oracle

4. Change permissions

[root@DevopsRoles ~]# chmod 750 /etc/init.d/oracle

5. use chkconfig the command to associate the dbora service

[root@DevopsRoles ~]# chkconfig --level 2345 oracle on

Test

restart the Oracle server. Then check the instance status

[oracle@DevopsRoles ~]$ ps -ef | grep smon | grep -v grep

Check the listener status

[oracle@DevopsRoles ~]$ lsnrctl status

Conclusion

Configuring Oracle for automatic startup on Linux significantly improves the efficiency and reliability of your database management. By following the steps outlined in this guide, you have learned how to set up your Oracle database to start automatically with your Linux system, ensuring minimal downtime and maximum productivity. Regular maintenance and monitoring will further ensure the smooth operation of your database. Keep exploring and optimizing your setup to make the most out of your Oracle database. Thank you for reading DevOpsRoles.com page

Linux

Vulnerability of sudo (CVE-2019-14287)

Leave a comment

Step by step Check Vulnerability of sudo CVE-2019-14287 in Linux. This vulnerability has been assigned CVE-2019-14287.

The important thing is that you can run as root even if root is explicitly prohibited.

My environment for Vulnerability of sudo

I have created account TestCVE14287 not permission sudo root.

TestCVE14287    ALL=(ALL,!root) ALL

Step by step Check Vulnerability sudo in Linux

You can see that the command can be executed if you do not root as shown below.

Run command in the picture as below:

Execute id command by TestCVE14287 itself

$ id

Execute whoami command with uid of TestCVE14287

$ sudo -u#7802 whoami

execute id command with uid = 1234

$ sudo -u#1234 id

execute id command with root privilege ( Input password of TestCVE14287)

$ sudo id

execute id command with uid = 0 (root)

$ sudo -u#0 whoami

However, How to vulnerability execute as root as the picture follows.

Run command in the picture as below:

Set -1 and execute id command

$ sudo -u#-1 id

Set 4294967295 and execute id command

$ sudo -u#4294967295 id

Set id and execute id command

$ sudo -u#-1 whoami

Set 4294967295 and execute whoami command

$ sudo -u#4294967295 whoami　

This makes it possible to execute with root privileges even if execution with root is explicitly prohibited by sudoers

Note:

  • PAM session module is not executed at runtime: -u the uid specified in the option does not exist in the password database

Conclusion

Better to use the latest version for security maintenance. Thank you for reading the DevopsRoles page!

Linux

How to use nmap command

Leave a comment

Introduction

In this tutorial, How to use NMAP Security Scanner on Linux. How to scan port, service name, and ping v.v to target.

The nmap command is a powerful network scanning tool used for discovering hosts and services on a computer network. It provides a wide range of options and functionalities.

Here are some common use cases of the nmap command:

Install Nmap

CentOS

$ sudo yum install nmap.

Debian

$ sudo apt-get install nmap.

Ubuntu

$ sudo apt-get install nmap.

Syntax use nmap command

nmap [Scan Type] [Options] {Target}

Target

For example, Domain: www.huuphan.com, www.devopsroles.com. IP Address: 192.168.1.4, 10.0.0-255.1-254

Scan Type

-sL:    List Scan -- simply list targets to scan
-sn:    Ping Scan -- disable port scan
-sP:    Ping Scan -- go no further than determining if host is online
-sS/sT/sA/sW/sM:    TCP SYN/Connect()/ACK/ Window/Maimon scans
-sV:    Probe open ports to determine service/version info
-sU:    UDP Scan
-sO:    IP protocol scan -- ICMP,EIGRP,
-b:    FTP bounce scan
-n/-R:    Never do DNS resolution/Always resolve[default: sometimes]

Options

-p: Only scan specified ports
-O: Enable OS detection
-P0: Treat all hosts as online -- skip host discovery
-e: Use specified interface

nmap command examples

Scan port

# all ports
$ nmap -v -PO www.huuphan.com

# speified port only
$ nmap -v -PO -p 22 192.168.1.0/24

# post scan & service name
$ nmap -v -sV 192.168.1.110

ping scan

$ nmap -v -sn 192.168.1.0/24

Check the OS of the specified IP

$ nmap -A 192.168.1.110

The nmap option –traceroute to trace the route from the scanning machine to the target host

$ nmap -Pn --traceroute -p 443 www.huuphan.com

Scan multiple hosts:

Specify multiple target IP addresses or hostnames separated by spaces.

Conclusion

Nmap command is a simple command in Linux. These are just a few examples of using the nmap command. It’s important to note that scanning hosts or networks without proper authorization may be illegal or against the terms of service.

Make sure you have the necessary permissions and comply with applicable laws and regulations before using nmap. It is a tips and tricks for troubleshooting Linux. Thank you for reading the DevopsRoles page!

Linux

nl command in Linux with example

Leave a comment

Introduction

In this tutorial, we will explore how to use the nl command in Linux to number lines of files. The nl command is a powerful tool for adding line numbers to the contents of a file or standard input. This can be incredibly useful for referencing specific lines more easily or for organizing and presenting content more clearly. Let’s delve into practical examples to demonstrate how the nl command can be effectively utilized in various scenarios.

What does the nl command mean?

The nl command stands for “number lines,” and it is used in Linux to add line numbers to the contents of files or standard input. This functionality is particularly useful for referencing specific lines more easily in scripts or documents.

nl command syntax

nl [OPTION]… [FILE]…

Some common options for the “nl” command include:

  • -b <type>: Specifies the numbering style. The <type> can be a (all lines), t (non-empty lines), or n (no lines).
  • -i <increment>: Sets the line number increment. The <increment> can be any positive integer.
  • -v <number>: Sets the starting line number. The <number> can be any positive integer.
  • -w <width>: Specifies the field width for line numbers.

On the man page, the describes it

  • nl – number lines of files.
  • man nl – More details information about nl command.

nl command in Linux with an example

I have created a file nl_command.txt as below

[vagrant@DevopsRoles ~]$ cat nl_command.txt                                    
 HuuPV, My website DevopsRoles.com and HuuPhan.com.SN:199x.
 Devops Roles.
 Devops Roles.
 Devops Roles.
 Devops Roles.
 Hello world :)

By default, the nl command doesn’t number empty lines

[vagrant@DevopsRoles ~]$ nl nl_command.txt 
      1  HuuPV, My website DevopsRoles.com and HuuPhan.com.SN:199x.
      2  Devops Roles.
      3  Devops Roles.
      4  Devops Roles.
      5  Devops Roles.
      6  Hello world :)

How to number empty lines.

[vagrant@DevopsRoles ~]$ nl -b a nl_command.txt                                
      1  HuuPV, My website DevopsRoles.com and HuuPhan.com.SN:199x.
      2  Devops Roles.
      3  Devops Roles.
      4  Devops Roles.
      5  Devops Roles.
      6  
      7  Hello world :)

How to numbering formats

[vagrant@DevopsRoles ~]$ nl -n ln nl_command.txt                               
 1       HuuPV, My website DevopsRoles.com and HuuPhan.com.SN:199x.
 2       Devops Roles.
 3       Devops Roles.
 4       Devops Roles.
 5       Devops Roles.
 6       Hello world :)
 [vagrant@DevopsRoles ~]$ nl -n rz nl_command.txt 
 000001  HuuPV, My website DevopsRoles.com and HuuPhan.com.SN:199x.
 000002  Devops Roles.
 000003  Devops Roles.
 000004  Devops Roles.
 000005  Devops Roles.
 000006  Hello world :)

Customized numbering separator

[vagrant@DevopsRoles ~]$ nl nl_command.txt                                     
      1  HuuPV, My website DevopsRoles.com and HuuPhan.com.SN:199x.
      2  Devops Roles.
      3  Devops Roles.
      4  Devops Roles.
      5  Devops Roles.
      6  Hello world :)
 [vagrant@DevopsRoles ~]$ nl -s : nl_command.txt 
      1:HuuPV, My website DevopsRoles.com and HuuPhan.com.SN:199x.
      2:Devops Roles.
      3:Devops Roles.
      4:Devops Roles.
      5:Devops Roles.
      6:Hello world :)

Conclusion

nl command is a simple command in Linux. It uses the number of lines of files. You can refer to the manual page for the “nl” command by typing man nl in the terminal for more information and additional options available on your specific Linux distribution. Thank you for reading the DevopsRoles page!

Linux

How to Use the tr Command in Linux: Examples Included

Leave a comment

Introduction

In this tutorial, we will explore the tr command in Linux, a versatile utility in Linux for translating or deleting characters in text. The tr command is primarily used for character-level transformations, making it an essential tool for modifying the contents of text files or input streams. By specifying certain rules, tr allows you to replace or remove specific characters efficiently. Let’s dive into practical examples to see the using tr command in Linux.

What is the purpose of the tr command?

The tr command, short for “translate,” is a powerful utility in Linux used to translate or delete characters in text data. It performs specific transformations on the contents of a file or data from a standard input, based on rules defined by the user.

tr command syntax

tr [OPTION]... [INPUT [OUTPUT]]

According to the man page, it is described as follows.

  • tr – translate or delete characters.
  • man tr – More details information about tr command.

tr command in Linux with an example

I have created a file tr_command.txt as below

[vagrant@DevopsRoles ~]$ cat tr_command.txt 
 HuuPV, My website DevopsRoles.com and HuuPhan.com.SN:199x.
 Devops Roles.
 Devops Roles.
 Devops Roles.
 Devops Roles.
 Hello world :)

How to convert small letters into capital letters.

[vagrant@DevopsRoles ~]$ cat tr_command.txt | tr [:lower:] [:upper:]           
 HUUPV, MY WEBSITE DEVOPSROLES.COM AND HUUPHAN.COM.SN:199X.
 DEVOPS ROLES.
 DEVOPS ROLES.
 DEVOPS ROLES.
 DEVOPS ROLES.
 HELLO WORLD :)

The following command is used to convert each space of the text by a newline (\n)

[vagrant@DevopsRoles ~]$ cat tr_command.txt | tr [:space:] '\n'                                                                                                            
 HuuPV,
 My
 website
 DevopsRoles.com
 and
 HuuPhan.com.SN:199x.
 Devops
 Roles.
 Devops
 Roles.
 Devops
 Roles.
 Devops
 Roles.
 Hello
 world
 :)

uses –s option for searching and replacing any string from a text

[vagrant@DevopsRoles ~]$ cat tr_command.txt |  tr -s ' ' '\t'
 HuuPV,  My      website DevopsRoles.com and     HuuPhan.com.SN:199x.
 Devops  Roles.
 Devops  Roles.
 Devops  Roles.
 Devops  Roles.
 Hello   world   :)

Delete numbers

[vagrant@DevopsRoles ~]$ cat tr_command.txt | tr -d "[:digit:]"
 HuuPV, My website DevopsRoles.com and HuuPhan.com.SN:x.
 Devops Roles.
 Devops Roles.
 Devops Roles.
 Devops Roles.
 Hello world :)

Output the converted output to a file

[vagrant@DevopsRoles ~]$ cat tr_command.txt | tr "[:upper:]" "[:lower:]" > result_tr.txt

Conclusion

The tr command is a straightforward yet powerful tool in Linux, used to translate or delete characters in text. For more detailed information and to explore additional options that may vary by Linux distribution, you can consult the manual page for the tr command by typing man tr in the terminal. Thank you for visiting the DevopsRoles page!

Linux

uniq command in Linux: A Guide to Eliminating Duplicate Lines

Leave a comment

Introduction

In this guide, I demonstrate how to use the uniq command in Linux to handle repeated lines. We’ll explore practical examples of the uniq command in action.

What does the “uniq” command mean?

“uniq” – This command is used to report or omit repeated lines.

uniq command the syntax

uniq [OPTION]... [INPUT [OUTPUT]]

In the man page, the describes it

  • uniq – report or omit repeated lines.
  • man uniq – More details information about uniq command.

How to Use Uniq Command in Linux

I have created a file uniq_command.txt as below

[vagrant@DevopsRoles ~]$ cat uniq_command.txt 
 HuuPV, My website DevopsRoles.com and HuuPhan.com.SN:199x.
 Devops Roles.
 Devops Roles.
 Devops Roles.
 Devops Roles.
 Hello world :)

Remove duplicate lines with the uniq command.

[vagrant@DevopsRoles ~]$ uniq uniq_command.txt 
 HuuPV, My website DevopsRoles.com and HuuPhan.com.SN:199x.
 Devops Roles.
 Hello world :)

The number of times a line was repeated

[vagrant@DevopsRoles ~]$ uniq -c uniq_command.txt 
       1 HuuPV, My website DevopsRoles.com and HuuPhan.com.SN:199x.
       4 Devops Roles.
       1 
       1 Hello world :)

It only prints the repeated lines.

[vagrant@DevopsRoles ~]$ uniq -d uniq_command.txt 
 Devops Roles.

Prints all repeated duplicate line

[vagrant@DevopsRoles ~]$ uniq -D uniq_command.txt 
 Devops Roles.
 Devops Roles.
 Devops Roles.
 Devops Roles.

How to not print the duplicate lines. Only the unique lines.

[vagrant@DevopsRoles ~]$ uniq -u uniq_command.txt 
 HuuPV, My website DevopsRoles.com and HuuPhan.com.SN:199x.
 Hello world :)

Conclusion

The uniq command is a straightforward tool in Linux, essential for managing and eliminating duplicate lines in files. It offers a simple yet effective way to clean data by reporting or omitting repeated entries. Thank you for visiting the DevopsRoles page and exploring this utility with us!

Linux

cut command in Linux with example

Leave a comment

Introduction

In this tutorial, I am using the cut command in Linux to remove sections from each line of files.

The cut the command is used to extract specific sections (columns) from lines of input text or files in Linux and Unix systems. It is particularly useful for working with delimited data.

What does cut command mean?

cut – remove sections from each line of files

Syntax

cut OPTION... [FILE]...

On the man page, the description it

  • cut – remove sections from each line of files.
  • man cut – More details information about the cut command.

cut command in Linux with an example

I have created a file cut_command.txt as below

[vagrant@DevopsRoles ~]$ cat cut_command.txt 
 HuuPV, My website DevopsRoles.com and HuuPhan.com.SN:199x.
 Devops Roles.
 Hello world. xxx.

For delimiter-separated fields. The default delimiter is the tab character.

[vagrant@DevopsRoles ~]$ cut -d "," -f 1 cut_command.txt 
 HuuPV
 Devops Roles.
 Hello world. xxx

Get a list of all users in the Linux

[vagrant@DevopsRoles ~]$ cat /etc/passwd | cut -d ':' -f1                                                                                                                                                                              
 root
 bin
 daemon
 adm
 lp
 sync
 shutdown
 halt
 mail
 uucp
 operator
 games
 gopher
 ftp
 nobody
 vcsa
 rpc
 rpcuser
 nfsnobody
 sshd
 exim
 centos
 huupv
 gluster
 grafana
 influxdb
 mysql
 acc1
 netdata
 telegraf
 nginx
 dbus
 haldaemon

cut and sort sort

[vagrant@DevopsRoles ~]$ cat /etc/passwd | grep home | cut -d: -f1,6 | sort
 acc1:/home/acc1
 huupv:/home/huupv

You can also display the table with column -t

[vagrant@DevopsRoles ~]$ cat /etc/passwd | grep home | cut -d: -f1,6 | sort | tr ":" " "  | column -t
 acc1      /home/acc1                                                           
 huupv    /home/huupv

Extract a range of fields

cut -f 2-4 file.txt

Print a specific delimiter

cut -d ',' -f 2 --output-delimiter=" | " file.csv

Read input from a pipe

echo "data1,data2,data3" | cut -d ',' -f 2

Conclusion

cut command in Linux is a simple command in Linux. It is used to remove sections from each line of files.

These are just a few examples of how you can use the cut command. It offers various options and functionalities for extracting specific sections from text or files. You can refer to the cut man page (man cut) for more details and additional options.

Thank you for reading the DevopsRoles page!

Devops Tutorial

Exit mobile version