Tag Archives: Linux

uniq command in Linux: A Guide to Eliminating Duplicate Lines

Introduction

In this guide, I demonstrate how to use the uniq command in Linux to handle repeated lines. We’ll explore practical examples of the uniq command in action.

What does the “uniq” command mean?

uniq” – This command is used to report or omit repeated lines.

uniq command the syntax

uniq [OPTION]... [INPUT [OUTPUT]]

In the man page, the describes it

  • uniq – report or omit repeated lines.
  • man uniq – More details information about uniq command.

How to Use Uniq Command in Linux

I have created a file uniq_command.txt as below

[vagrant@DevopsRoles ~]$ cat uniq_command.txt 
 HuuPV, My website DevopsRoles.com and HuuPhan.com.SN:199x.
 Devops Roles.
 Devops Roles.
 Devops Roles.
 Devops Roles.
 Hello world :)

Remove duplicate lines with the uniq command.

[vagrant@DevopsRoles ~]$ uniq uniq_command.txt 
 HuuPV, My website DevopsRoles.com and HuuPhan.com.SN:199x.
 Devops Roles.
 Hello world :)

The number of times a line was repeated

[vagrant@DevopsRoles ~]$ uniq -c uniq_command.txt 
       1 HuuPV, My website DevopsRoles.com and HuuPhan.com.SN:199x.
       4 Devops Roles.
       1 
       1 Hello world :)

It only prints the repeated lines.

[vagrant@DevopsRoles ~]$ uniq -d uniq_command.txt 
 Devops Roles.

Prints all repeated duplicate line

[vagrant@DevopsRoles ~]$ uniq -D uniq_command.txt 
 Devops Roles.
 Devops Roles.
 Devops Roles.
 Devops Roles.

How to not print the duplicate lines. Only the unique lines.

[vagrant@DevopsRoles ~]$ uniq -u uniq_command.txt 
 HuuPV, My website DevopsRoles.com and HuuPhan.com.SN:199x.
 Hello world :)

Conclusion

The uniq command is a straightforward tool in Linux, essential for managing and eliminating duplicate lines in files. It offers a simple yet effective way to clean data by reporting or omitting repeated entries. Thank you for visiting the DevopsRoles page and exploring this utility with us!

fmt command in Linux: A Practical Guide

Introduction

In this tutorial, we’ll explore how to use the fmt command in Linux to neatly format text in files, set optimal column widths, and standardize spacing. This guide aims to enhance your text processing skills on Linux using practical examples of the fmt command. Let’s dive into the capabilities of the fmt command in Linux.

What does the fmt command mean?

fmt – a straightforward command used as an optimal text formatter.

Syntax

fmt [-WIDTH] [OPTION]... [FILE]...


In the manual page, the fmt command is described as a simple and optimal text formatting Linux. For more detailed information about the fmt command, you can refer to the man fmt section in the manual.

fmt command in Linux with an example

By default, fmt sets the column width at 75.

I have created a file fmt_command.txt

[vagrant@DevopsRoles ~]$ cat fmt_command.txt 
 HuuPV, My website DevopsRoles.com and HuuPhan.com.SN:199x.
 Devops Roles.
 Hello world. xxx.

fmt with no options

[vagrant@DevopsRoles ~]$ fmt fmt_command.txt 
HuuPV, My website DevopsRoles.com and HuuPhan.com.SN:199x.  Devops Roles.

Hello world. xxx.

How to change the width of formatting.

[vagrant@DevopsRoles ~]$ fmt --width 20 fmt_command.txt 
 HuuPV, My website
 DevopsRoles.com and
 HuuPhan.com.SN:199x.
 Devops Roles.
 Hello world. xxx.

with -u option uses one space between words and two spaces after sentences for formatting.

[vagrant@DevopsRoles ~]$ fmt -u fmt_command.txt 
 HuuPV, My website DevopsRoles.com and HuuPhan.com.SN:199x.  Devops Roles.
 Hello world. xxx.

Formatting Multiple Files

You can also format multiple files simultaneously. Simply list the files as arguments:

fmt file1.txt file2.txt file3.txt

fmt will process each file and output the formatted text for all of them.

Splitting Long Lines

If you need to split long lines without breaking words, use the -s or --split-only option:

fmt -s myfile.txt

This option splits lines at spaces, ensuring words are not cut off.

Advanced Formatting Options

The fmt command offers several advanced options for more precise text formatting:

  • -c or --crown-margin: Preserve the indentation of the first two lines.
  • -t or --tagged-paragraph: Format the text as tagged paragraphs, maintaining a hanging indentation.
  • -g or --goal: Set the goal width for formatting. This is the preferred width, while the -w option sets the maximum width.

Frequently Asked Questions

What is the default width for the fmt command?

The default width is 75 characters. You can change it using the -w or --width option.

Can fmt handle multiple files at once?

Yes, you can list multiple files as arguments and fmt will format each of them.

How can I preserve indentation with fmt?

Use the -u or --uniform-spacing option to keep the original indentation intact.

What is the difference between the -w and -g options?

The -w option sets the maximum line width, while the -g option sets the goal width, which is the preferred width for formatting.

Can fmt split long lines without breaking words?

Yes, use the -s or --split-only option to split lines at spaces without cutting off words.

Conclusion

The fmt command is a straightforward Linux utility that helps format text files, set column width commands, and ensure uniform spacing. It simplifies text editing tasks, making it a valuable tool for developers and system administrators alike. Thank you for reading this guide on the DevopsRoles page!

By understanding and utilizing the various options and features of fmt, you can efficiently format text files to meet your specific requirements. Experiment with the examples provided in this guide and explore the full potential of the fmt command in your Linux environment.

Install Gradle on CentOS

In this tutorial, How to install Gradle on CentOS. Gradle is a free and Open Source that helps your build, automate and deliver better software, faster.

Prerequisites

  • Server instance : CentOS 7
  • A sudo user.

Your System update

Login into the sudo user and run the following commands to update your system.

$ sudo yum -y install epel-release
$ sudo yum -y update
$ sudo reboot

Install JDK

Gradle requires Java Development Kit (JDK) 7 or higher in order to work. Link here

Download Gradle

In this guide, I use the “binary-only” archive. Link the Gradle release page to the latest version of Gradle. Using wget command to download Gradle.

$ cd /opt/
$ wget https://downloads.gradle-dn.com/distributions/gradle-5.6.2-bin.zip

Install Gradle on CentOS

Your run the command following.

$ sudo mkdir /opt/gradle
$ sudo unzip -d /opt/gradle gradle-5.6.2-bin.zip

Set the PATH environment for Gradle executable.

$ export PATH=$PATH:/opt/gradle/gradle-5.6.2/bin

To check if the Gradle install was successful.

$ gradle -v

Your system is now built the program with Gradle. Thank you for reading the DevopsRoles page!

How to Install and configure ssh server on Centos 7

Introduction

In this tutorial, we will install and configure the SSH server on CentOS 7. SSH, also known as Secure Socket Shell, is a network protocol that allows for secure remote login from one computer to another. It provides system administrators a secure way to access a server remotely.

Prerequisites

Before we start, ensure you have:

  • A CentOS 7 server with a non-root user having sudo privileges.
  • Internet access to download the necessary packages.

Install and configure SSH server on Centos 7

Installing SSH Server

Step 1: Update Your System

First, update your system to ensure all existing packages are up-to-date:

sudo yum update -y

Step 2: Install OpenSSH Server

Install OpenSSH, the most popular SSH server package:

sudo yum install -y openssh openssh-server openssh-clients openssl-libs

Starting and Enabling SSH Service

Once the installation is complete, start the SSH service and enable it to start on boot.

Step 3: Start SSH Service

Start the SSH service using the command:

sudo systemctl start sshd

Step 4: Enable SSH Service

Enable the SSH service to start automatically on system boot:

sudo systemctl enable sshd

Step 5: Check SSH Service Status

Verify the SSH service status with:

sudo systemctl status sshd

Configuring SSH Server

Basic Configuration

The SSH server configuration file is located at /etc/ssh/sshd_config. You can edit this file to customize the SSH server settings.

Step 6: Open SSH Configuration File

Open the SSH configuration file with a text editor:

sudo vi /etc/ssh/sshd_config

Step 7: Disable Root Login

For security reasons, it is recommended to disable root login. Find and change the following line:

PermitRootLogin no

Step 8: Save and Exit

Save the changes and exit the editor. In vi, you can do this by pressing Esc, typing :wq, and hitting Enter.

Step 9: Restart SSH Service

After making the changes, restart the SSH service:

sudo systemctl restart sshd

Step 10: Allow SSH Through the Firewall

If Firewalld is running, allow SSH port 22/tcp:

sudo firewall-cmd --add-service=ssh --permanent
sudo firewall-cmd --reload

Advanced Configuration

For advanced users, additional configuration options can further secure your SSH server.

Step 11: Configure Public Key Authentication

Public key authentication is more secure than password authentication. Ensure the following lines are set in the configuration file:

PubkeyAuthentication yes
AuthorizedKeysFile .ssh/authorized_keys

Step 12: Configure Two-Factor Authentication

Two-factor authentication adds an extra layer of security. You can set it up by installing and configuring google-authenticator.

sudo yum install google-authenticator -y google-authenticator

Follow the prompts to set up two-factor authentication.

Step 13: Restrict SSH Access by IP Address

You can limit SSH access to specific IP addresses by adding the following lines:

AllowUsers yourusername@192.168.1.100
DenyUsers baduser@192.168.1.*

Configuring SSH Client

Step 14: Install SSH Client

To connect to the SSH server, install the SSH client:

sudo yum -y install openssh-clients

Step 15: Connect to SSH Server

Use a common user to connect to the SSH server:

ssh yourusername@your_server_ip

Replace yourusername with your actual username and your_server_ip with the server’s IP address.

Transferring Files Using SSH

Using SCP (Secure Copy)

SCP allows for secure file transfer between the local machine and the remote server.

Step 16: Copy the File to the Remote Server

scp ./localfile.txt yourusername@your_server_ip:/remote/directory/

Step 17: Copy the File from the Remote Server

scp yourusername@your_server_ip:/remote/directory/remotefile.txt ./localdirectory/

Using SFTP (SSH File Transfer Protocol)

SFTP is another method for secure file transfer. It is typically enabled by default.

Step 18: Connect to SFTP

sftp yourusername@your_server_ip

Step 19: Common SFTP Commands

  • Show the current directory on the remote server: pwd
  • Show the current directory on the local server: !pwd
  • List files in the current directory on the remote server: ls -l
  • List files in the current directory on the local server: !ls -l
  • Change the directory on the remote server: cd /remote/directory/
  • Upload a file to the remote server: put localfile.txt remotefile.txt
  • Download a file from the remote server: get remotefile.txt localfile.txt
  • Delete a directory on the remote server: rmdir directoryname
  • Delete a file on the remote server: rm filename
  • Execute commands on the local server: !command
  • Exit SFTP: quit

SSH Key-Pair Authentication

Step 20: Create SSH Key Pair

Generate a new SSH key pair on the client machine:

ssh-keygen -t rsa

Step 21: Move the Public Key to the Authorized Keys

Move the generated public key to the server’s authorized keys file:

mv ~/.ssh/id_rsa.pub ~/.ssh/authorized_keys

Step 22: Secure the .ssh Directory

mkdir ~/.ssh
chmod 700 ~/.ssh

Step 23: Transfer the Secret Key to the Client

Copy the secret key from the server to the client’s SSH directory:

scp yourusername@your_server_ip:/home/yourusername/.ssh/id_rsa ~/.ssh/

Step 24: Connect Using SSH Key

ssh -i ~/.ssh/id_rsa yourusername@your_server_ip

Step 25: Disable Password Authentication

Disable password authentication for enhanced security. Edit the SSH configuration file:

PasswordAuthentication no
ChallengeResponseAuthentication no
UsePAM yes

Restart the SSH service:

sudo systemctl restart sshd

Common Issues and Troubleshooting

Issue 1: Connection Refused

If you encounter a “Connection refused” error, check if the SSH service is running and the firewall settings allow SSH traffic:

sudo systemctl status sshd
sudo firewall-cmd --permanent --add-service=ssh
sudo firewall-cmd --reload

Issue 2: Permission Denied

If you see a “Permission denied” error, ensure your user has the correct permissions and the SSH keys are correctly configured.

Issue 3: SSH Service Not Starting

If the SSH service fails to start, check the configuration file for syntax errors using:

sudo sshd -t

FAQs

Q: How do I restart the SSH service?

A: You can restart the SSH service using:

sudo systemctl restart sshd

Q: How do I check the SSH server version?

A: Check the SSH server version with:

ssh -V

Q: Can I use SSH keys for authentication?

A: Yes, SSH keys provide a secure way of authentication. Follow the steps in the advanced configuration section to set it up.

Conclusion

Setting up and configuring an SSH server on CentOS 7 is a crucial skill for system administrators. This guide covered everything from basic installation to advanced configuration, ensuring your SSH server is secure and efficient. By following these steps, you can enhance your server’s security and manage it remotely with ease. Thank you for reading the DevopsRoles page!

Step-by-Step: Install and configure ntp centos 7

Introduction

In this tutorial, How to install and configure NTP server in Centos. NTP- is a protocol that runs over port 123 UDP at the Transport Layer and allows computers to synchronize time over networks for an accurate time.

What is NTP?

NTP, or Network Time Protocol, is a networking protocol designed to synchronize the clocks of computers to a reference time source. It is used in various networked environments to ensure that all systems maintain accurate time, which is essential for tasks such as logging, security, and scheduled operations.

Why Use NTP on CentOS 7?

Using NTP on CentOS 7 helps you:

  • Maintain accurate system time across all servers.
  • Ensure the proper functioning of time-dependent applications.
  • Avoid issues caused by time discrepancies.

Prerequisites

Before you begin, make sure you have the following:

  • A CentOS 7 server with root or sudo access.
  • A stable internet connection to access NTP servers.

Install and configure NTP

[vagrant@DevopsRoles ~]$ sudo yum -y install ntp

Configure NTP server

[vagrant@DevopsRoles ~]$ sudo vi /etc/ntp.conf

The content file “ntp.conf” as below

# line 18: add the network range your network
restrict 10.0.2.0 mask 255.255.255.0 nomodify notrap
# change servers for synchronization
server 0.asia.pool.ntp.org
server 1.asia.pool.ntp.org
server 2.asia.pool.ntp.org
server 3.asia.pool.ntp.org

Start and startup ntp

[vagrant@DevopsRoles ~]$ sudo systemctl start ntpd
[vagrant@DevopsRoles ~]$ sudo systemctl enable ntpd

If Firewalld is running, allow NTP service. NTP uses 123/UDP.

[vagrant@DevopsRoles ~]$ sudo firewall-cmd --add-service=ntp --permanent
[vagrant@DevopsRoles ~]$ sudo firewall-cmd --reload

Check it works normally

[vagrant@DevopsRoles ~]$ ntpq -p

Conclusion

Installing and configuring NTP on CentOS 7 is essential for maintaining accurate system time, which is critical for various applications and services. By following this guide, you can ensure that your CentOS 7 server is properly synchronized with reliable time sources. Whether you are managing a single server or a complex network, NTP provides the accuracy and reliability needed for effective time management.

By carefully configuring NTP and addressing any potential issues, you can maintain a robust and accurate time synchronization setup, ensuring smooth and efficient operations across your network. I hope will this your helpful. Thank you for reading the DevopsRoles page!

Mastering the Netstat Command in Linux: A Comprehensive Guide with Examples

Introduction

Welcome to this tutorial where I’ll guide you through the basics to advanced uses of the netstat command in Linux, with practical examples to help you master this tool.

What is Netstat?

Netstat is a command-line utility used to display all active network connections, both incoming and outgoing, on Unix, Linux, and Windows NT-based systems. It’s invaluable for network administration and monitoring.

Details can be found on the netstat command manual page:

[root@DevopsRoles ~]# man netstat | more

Detailed Usage of the Netstat Command in Linux

Here’s how you can use the netstat command in Linux to explore various network statistics:

1. List all LISTENING Ports of TCP and UDP connections using netstat -a option

2. Viewing Open TCP Socket Connections

This displays all active TCP connections. Execute this command to see detailed socket information.

[root@DevopsRoles ~]# netstat -nplt
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 127.0.0.1:8088          0.0.0.0:*               LISTEN      659/influxd
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      792/master
tcp        0      0 0.0.0.0:111             0.0.0.0:*               LISTEN      319/rpcbind
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      655/sshd
tcp6       0      0 :::3000                 :::*                    LISTEN      662/grafana-server
tcp6       0      0 ::1:25                  :::*                    LISTEN      792/master
tcp6       0      0 :::111                  :::*                    LISTEN      319/rpcbind
tcp6       0      0 :::8086                 :::*                    LISTEN      659/influxd
tcp6       0      0 :::22                   :::*                    LISTEN      655/sshd

3. Viewing Open UDP Socket Connections

Similar to TCP, this command shows all UDP connections currently open and active.

[root@DevopsRoles ~]# netstat -nplu
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
udp        0      0 127.0.0.1:323           0.0.0.0:*                           313/chronyd
udp        0      0 0.0.0.0:68              0.0.0.0:*                           464/dhclient
udp        0      0 0.0.0.0:111             0.0.0.0:*                           319/rpcbind
udp        0      0 0.0.0.0:906             0.0.0.0:*                           319/rpcbind
udp6       0      0 ::1:323                 :::*                                313/chronyd
udp6       0      0 :::111                  :::*                                319/rpcbind
udp6       0      0 :::906                  :::*                                319/rpcbind

4. List all TCP Listening Ports

[root@DevopsRoles ~]# netstat -lt
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State
tcp        0      0 DevopsRoles:radan-http  0.0.0.0:*               LISTEN
tcp        0      0 DevopsRoles:smtp        0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:sunrpc          0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:ssh             0.0.0.0:*               LISTEN
tcp6       0      0 [::]:hbci               [::]:*                  LISTEN
tcp6       0      0 localhost:smtp          [::]:*                  LISTEN
tcp6       0      0 [::]:sunrpc             [::]:*                  LISTEN
tcp6       0      0 [::]:d-s-n              [::]:*                  LISTEN
tcp6       0      0 [::]:ssh                [::]:*                  LISTEN

4. List all UDP Listening Ports

[root@DevopsRoles ~]# netstat -lu
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State
udp        0      0 DevopsRoles:323         0.0.0.0:*
udp        0      0 0.0.0.0:bootpc          0.0.0.0:*
udp        0      0 0.0.0.0:sunrpc          0.0.0.0:*
udp        0      0 0.0.0.0:906             0.0.0.0:*
udp6       0      0 localhost:323           [::]:*
udp6       0      0 [::]:sunrpc             [::]:*
udp6       0      0 [::]:906                [::]:*

5. Show Statistics by Protocol

[root@DevopsRoles ~]# netstat -s
Ip:
    9607 total packets received
    0 forwarded
    0 incoming packets discarded
    9605 incoming packets delivered
    4614 requests sent out
    7 outgoing packets dropped
Icmp:
    16 ICMP messages received
    0 input ICMP message failed.
    ICMP input histogram:
        destination unreachable: 16
    16 ICMP messages sent
    0 ICMP messages failed
    ICMP output histogram:
        destination unreachable: 16
IcmpMsg:
        InType3: 16
        OutType3: 16
Tcp:
    267 active connections openings
    412 passive connection openings
    2 failed connection attempts
    3 connection resets received
    3 connections established
    20699 segments received
    19546 segments send out
    66 segments retransmited
    0 bad segments received.
    13 resets sent
Udp:
    184 packets received
    16 packets to unknown port received.
    0 packet receive errors
    200 packets sent
    0 receive buffer errors
    0 send buffer errors
UdpLite:
TcpExt:
    255 TCP sockets finished time wait in fast timer
    245 delayed acks sent
    16 delayed acks further delayed because of locked socket
    Quick ack mode was activated 66 times
    6400 packet headers predicted
    2503 acknowledgments not containing data payload received
    8067 predicted acknowledgments
    TCPLossProbes: 66
    TCPLossProbeRecovery: 65
    66 DSACKs sent for old packets
    66 DSACKs received
    TCPDSACKIgnoredNoUndo: 65
    TCPRcvCoalesce: 3322
    TCPOrigDataSent: 14558
    TCPHystartTrainDetect: 7
    TCPHystartTrainCwnd: 124
IpExt:
    InNoRoutes: 2
    InOctets: 1806054
    OutOctets: 7957156
    InNoECTPkts: 9899

6. Displaying Service name with PID

[root@DevopsRoles ~]# netstat -tp
Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 DevopsRoles:ssh         gateway:52836           ESTABLISHED 2434/sshd: vagrant
tcp6       0      0 localhost:44918         localhost:d-s-n         ESTABLISHED 654/telegraf
tcp6       0      0 localhost:d-s-n         localhost:44918         ESTABLISHED 659/influxd

7. Displaying Promiscuous Mode

[root@DevopsRoles ~]# netstat -ac 6 | grep tcp
tcp        0      0 DevopsRoles:radan-http  0.0.0.0:*               LISTEN
tcp        0      0 DevopsRoles:smtp        0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:sunrpc          0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:ssh             0.0.0.0:*               LISTEN
tcp        0      0 DevopsRoles:ssh         gateway:52836           ESTABLISHED
tcp6       0      0 [::]:hbci               [::]:*                  LISTEN
tcp6       0      0 localhost:smtp          [::]:*                  LISTEN
tcp6       0      0 [::]:sunrpc             [::]:*                  LISTEN
tcp6       0      0 [::]:d-s-n              [::]:*                  LISTEN
tcp6       0      0 [::]:ssh                [::]:*                  LISTEN
tcp6       0      0 localhost:44918         localhost:d-s-n         ESTABLISHED
tcp6       0      0 localhost:d-s-n         localhost:44918         ESTABLISHED

8. Show Network Interface Transactions

[root@DevopsRoles ~]# netstat -i
Kernel Interface table
Iface             MTU    RX-OK RX-ERR RX-DRP RX-OVR    TX-OK TX-ERR TX-DRP TX-OVR Flg
eth0             1500    10190      0      0 0          8724      0      0      0 BMRU
lo              65536    12237      0      0 0         12237      0      0      0 LRU

9. Find Listening Programs

[root@DevopsRoles ~]# netstat -ap | grep grafana
tcp6       0      0 [::]:hbci               [::]:*                  LISTEN      662/grafana-server
unix  3      [ ]         STREAM     CONNECTED     14247    662/grafana-server

Conclusion

By following the netstat examples provided, you can effectively leverage the netstat command in Linux to gain insights into your system’s network connections. This guide aims to be a practical resource for both new and seasoned users. Thank you for choosing DevopsRoles for your learning needs!

Influxdb getting started

Introduction

In this tutorial, we get started with InfluxDB. we can use commands with InfluxDB. In the latter-mentioned post, I created a “telegraf” database in InfluxDB.

Let’s know if InfluxDB getting started

Now jump into InfluxDB.

[root@devopsroles.localhost ~]# influx
Connected to http://localhost:8086 version 1.7.4
InfluxDB shell version: 1.7.4
Enter an InfluxQL query
> 

View a list of all the databases using the “show databases” command.

> show databases                                                                                                                                                                                                                             
name: databases
name
----
_internal
netdata
monitoring
telegraf
opentsdb
> 

“_internal” is an internal InfluxDB database. To use telegraf database the “use telegraf” command:

> use telegraf                                                                                                                                                                                                                               
Using database telegraf

Now we are inside of the telegraf database. using the “show measurements” command:

> show measurements                                                                                                                                                                                                                          
name: measurements
name
----
cpu
disk
diskio
kernel
mem
processes
swap
system

Field Keys in the telegraf database.

> show field keys                                                                                                                                                                                                                            
name: cpu
fieldKey         fieldType
--------         ---------
usage_guest      float
usage_guest_nice float
usage_idle       float
usage_iowait     float
usage_irq        float
usage_nice       float
usage_softirq    float
usage_steal      float
usage_system     float
usage_user       float

name: disk
fieldKey     fieldType
--------     ---------
free         integer
inodes_free  integer
inodes_total integer
inodes_used  integer
total        integer
used         integer
used_percent float

name: diskio
fieldKey         fieldType
--------         ---------
io_time          integer
iops_in_progress integer
read_bytes       integer
read_time        integer
reads            integer
weighted_io_time integer
write_bytes      integer
write_time       integer
writes           integer

name: kernel
fieldKey         fieldType
--------         ---------
boot_time        integer
context_switches integer
entropy_avail    integer
interrupts       integer
processes_forked integer

name: mem
fieldKey          fieldType
--------          ---------
active            integer
available         integer
available_percent float
buffered          integer
cached            integer
commit_limit      integer
committed_as      integer
dirty             integer
free              integer
high_free         integer
high_total        integer
huge_page_size    integer
huge_pages_free   integer
huge_pages_total  integer
inactive          integer
low_free          integer
low_total         integer
mapped            integer
page_tables       integer
shared            integer
slab              integer
swap_cached       integer
swap_free         integer
swap_total        integer
total             integer
used              integer
used_percent      float
vmalloc_chunk     integer
vmalloc_total     integer
vmalloc_used      integer
wired             integer
write_back        integer
write_back_tmp    integer

name: processes
fieldKey      fieldType
--------      ---------
blocked       integer
dead          integer
idle          integer
paging        integer
running       integer
sleeping      integer
stopped       integer
total         integer
total_threads integer
unknown       integer
zombies       integer

name: swap
fieldKey     fieldType
--------     ---------
free         integer
in           integer
out          integer
total        integer
used         integer
used_percent float

name: system
fieldKey      fieldType
--------      ---------
load1         float
load15        float
load5         float
n_cpus        integer
n_users       integer
uptime        integer
uptime_format string

Tag Keys in the telegraf database.

> show tag keys                                                                                                                                                                                                                              
name: cpu
tagKey
------
cpu
host

name: disk
tagKey
------
device
fstype
host
mode
path

name: diskio
tagKey
------
host
name

name: kernel
tagKey
------
host

name: mem
tagKey
------
host

name: processes
tagKey
------
host

name: swap
tagKey
------
host

name: system
tagKey
------
host

InfluxDB Queries

How fields and tags work together. For example as below

> select * from cpu where time > now() - 10s                                                                                                                                                                                                 
name: cpu
time                cpu       host                         usage_guest usage_guest_nice usage_idle       usage_iowait      usage_irq usage_nice usage_softirq usage_steal usage_system        usage_user
----                ---       ----                         ----------- ---------------- ----------       ------------      --------- ---------- ------------- ----------- ------------        ----------
1557722520000000000 cpu-total devopsroles.localhost 0           0                92.4924924924855 6.106106106081209 0         0          0             0           0.30030030030030713 1.10110110109885
1557722520000000000 cpu0      devopsroles.localhost 0           0                92.4924924924855 6.106106106081209 0         0          0             0           0.30030030030030713 1.10110110109885

> select * from cpu where cpu='cpu-total' and host='devopsroles.localhost' and time > now() - 10s                                                                                                                                     
name: cpu
time                cpu       host                         usage_guest usage_guest_nice usage_idle       usage_iowait      usage_irq usage_nice usage_softirq usage_steal usage_system        usage_user
----                ---       ----                         ----------- ---------------- ----------       ------------      --------- ---------- ------------- ----------- ------------        ----------
1557727830000000000 cpu-total devopsroles.localhost 0           0                96.7967967967239 2.402402402420665 0         0          0             0           0.20020020020020476 0.6006006006006143

> select usage_user,cpu,host from cpu where cpu='cpu-total' and host='devopsroles.localhost' and time > now() - 20s                                                                                                                   
name: cpu
time                usage_user         cpu       host
----                ----------         ---       ----
1557727930000000000 1.6016016016016381 cpu-total devopsroles.localhost

Series is a “collection of data in InfluxDB’s data structure that share a measurement, tag set, and retention policy.” Thank you for reading the DevopsRoles page!

Influxdb getting started. Influxdb getting started. Influxdb getting started.

How to install Telegraf on Linux

Introduction

Telegraf is a versatile and efficient agent for collecting and reporting metrics. It supports a wide array of data stores such as InfluxDB, Graphite, OpenTSDB, Datadog, and many more. This guide will walk you through the process of installing Telegraf on various Linux distributions, including Ubuntu and CentOS, ensuring you can get started with minimal hassle. How to install Telegraf on Linux is a step below.

Features of Telegraf

  • Supports Multiple Datastores: Compatible with InfluxDB, Graphite, OpenTSDB, Datadog, and many others.
  • Plugin-Driven: Easily extendable through numerous input and output plugins.
  • Small Memory Footprint: Designed to be lightweight and efficient.

Prerequisites

Before proceeding with the installation, ensure your system meets the following requirements:

  • A Linux distribution (Ubuntu, CentOS, Debian, etc.)
  • Root or sudo access to the system
  • Internet connection for downloading Telegraf packages

Installation Steps

RedHat & CentOS

To install Telegraf on RedHat and CentOS systems, follow these steps:

Download the Telegraf RPM package.

wget https://dl.influxdata.com/telegraf/releases/telegraf-1.10.3-1.x86_64.rpm

Install the downloaded package using yum.

sudo yum localinstall telegraf-1.10.3-1.x86_64.rpm

Verify the installation by checking the Telegraf version.

telegraf -version The output should be similar to:lessCopy codeTelegraf 1.10.3 (git: HEAD 294bb666)

Ubuntu & Debian

For Debian-based systems such as Ubuntu, follow these steps:

Download the Telegraf DEB package.

wget https://dl.influxdata.com/telegraf/releases/telegraf_1.10.3-1_amd64.deb

Install the downloaded package using dpkg.

sudo dpkg -i telegraf_1.10.3-1_amd64.deb

OS X (via Homebrew)

For macOS users, Telegraf can be installed using Homebrew:

  1. Update Homebrew. brew update
  2. Install Telegraf. brew install telegraf

Starting the Telegraf Service

After installing Telegraf, you need to start and enable the service.

  1. Start the Telegraf service. sudo systemctl start telegraf
  2. Enable the Telegraf service to start on boot. sudo systemctl enable telegraf

Configuration

Telegraf’s configuration file is located at /etc/telegraf/telegraf.conf. This file defines how Telegraf collects and outputs data.

Basic Configuration

Open the configuration file in your preferred text editor.

sudo nano /etc/telegraf/telegraf.conf

Input Plugins

Enable the CPU input plugin by adding the following configuration:

[[inputs.cpu]]
percpu = true
totalcpu = true
collect_cpu_time = false
report_active = false

Output Plugins

To output data to InfluxDB, configure the output plugin as follows:

[[outputs.influxdb]]
urls = ["http://localhost:8086"]
database = "telegraf"
retention_policy = ""
write_consistency = "any"
timeout = "5s"

Advanced Configuration

Adding Custom Plugins

You can extend Telegraf’s functionality by adding custom plugins. Place your custom plugin scripts in the appropriate directory and reference them in the configuration file.

Using Environment Variables

Telegraf supports environment variables, which can be useful for managing configurations in different environments.

Securing Telegraf

Ensure Telegraf is secure by:

  • Running it with the least privileges necessary.
  • Using HTTPS for data transmission.
  • Regularly updating to the latest version.

Troubleshooting

Common Issues

  • Service not starting: Check the system logs for errors using journalctl -u telegraf.
  • Data not collected: Ensure the input plugins are correctly configured.
  • Data not sent: Verify the output plugin configuration and network connectivity.

Useful Commands

  • Check the configuration: telegraf --config /etc/telegraf/telegraf.conf --test
  • View logs: sudo journalctl -u telegraf

Frequently Asked Questions (FAQs)

What is Telegraf used for?

Telegraf is used for collecting, processing, and writing metrics and events from various sources to various outputs.

Can Telegraf run on Windows?

Yes, Telegraf is cross-platform and can run on Windows, macOS, and Linux.

How do I update Telegraf?

To update Telegraf, simply follow the installation steps again, as the package manager will handle the upgrade process.

Is Telegraf free to use?

Yes, Telegraf is open-source and free to use under the MIT license.

Conclusion

Installing Telegraf on Linux is a straightforward process when you follow the right steps. From basic installation to advanced configuration, this guide has covered everything you need to get started with Telegraf. By leveraging its powerful features, you can efficiently collect and manage metrics for your infrastructure. Whether you’re a beginner or an advanced user, Telegraf offers the flexibility and scalability needed for effective monitoring.

Start your journey with Telegraf today and ensure your system’s performance is always at its peak! Thank you for reading the DevopsRoles page!

Install nslookup on Linux

Introduction

In this tutorial, How to install on Linux. is part of the bind-utils package. The package bind-utils is not yet installed on Linux, then you type command not found on Linux.

To install nslookup on Linux, you need to install the dnsutils package, which contains the nslookup utility. The process for installing the package can vary depending on the Linux distribution you are using. Here are the commands for some popular distributions:

How to Install nslookup on Linux

Install nslookup for Centos

[vagrant@DevopsRoles ~]$ sudo yum install bind-utils

Install nslookup for Ubuntu

Use apt-cache to search the package for nslookup command.

[vagrant@DevopsRoles ~]$ apt-cache search nslookup

The result found 2 packages that are related to nslookup.

dnsutils - Clients provided with BIND
libnet-nslookup-perl - simple DNS lookup module for perl

so install nslookup

[vagrant@DevopsRoles ~]$ sudo apt-get install dnsutils

Once the installation is complete, you can use nslookup from the command line.

[vagrant@DevopsRoles ~]$ nslookup x.x.x.x

Conclusion

In this guide, we have covered how to install and use the nslookup command on Linux, specifically for CentOS and Ubuntu. Whether you’re troubleshooting DNS issues or simply querying domain name records, nslookup is a powerful and essential tool for network administrators and IT professionals.

By following the simple installation steps, you can quickly enable nslookup on your system and start resolving domain names with ease. If you encounter any issues, ensure that your system’s package manager is up to date and that you have the necessary permissions to install packages.

We hope this guide has been helpful! If you have any questions or need further assistance, feel free to leave a comment or check out our other Linux tutorials. Thank you for reading the DevopsRoles page!

Mastering the gunzip command in Linux: A Comprehensive Guide with Examples

Introduction

gunzip command in Linux, compressing and decompressing files are essential skills for users. One of the most popular commands for decompressing files in Linux is gunzip. This command helps decompress files that have been compressed using gzip, a common compression format in Unix-like systems. In this article, we will delve into the details of the gunzip command, how to use it, and provide practical examples that you can apply in your daily tasks.

Syntax

gunzip [ -acfhlLnNrtvV ] [-S suffix] [ name … ]

According to the man page, the gunzip command is used to compress or expand files.

To get more detailed information about the gunzip command, you can use:

man gunzip

gunzip command in Linux with Examples

$ gunzip devopsroles.txt.gz

Keep both the compressed and Decompressed files.

$ gunzip -k devopsroles.txt.gz

Display compressed within it without decompressing first.

$ gunzip -c devopsroles.txt.gz

Test Whether a Compressed File Is Valid before Decompressing it.

$ gunzip -t devopsroles.txt.gz

Show verbose information when you decompress the file.

$ gunzip -v devopsroles.txt.gz

Decompress Multiple Files at Once

gunzip file1.gz file2.gz file3.gz

To decompress a file while keeping the original compressed file, use the -c option and redirect the output:

gunzip -c file.gz > file

Conclusion

The gunzip command is a powerful and easy-to-use tool in Linux for decompressing gzip files. By mastering its options and syntax, you can save time and effort in file management. Hopefully, this article has provided you with a clearer understanding of how to use gunzip command in Linux effectively in your daily tasks. Keep exploring and leveraging the powerful tools of Linux to enhance your work efficiency and system management. Thank you for reading the DevopsRoles page!