How to Secure a Linux Server: Step-by-Step Guide

Running a Linux server gives you flexibility and control over your websites, applications, databases, and hosting environment. But simply installing Linux doesn’t mean your server is completely secure.

Weak passwords, outdated software, unnecessary open ports, and poorly configured applications can leave your server exposed to security threats. The good news is that you can reduce many of these risks by following a few practical security measures.

In this guide, we’ll explain how to secure a Linux server step by step, including useful commands, configuration examples, and tips to help you protect your server without making the process unnecessarily complicated.

What “Secure” Means for a VPS

No server is unhackable, and no single tool makes it so. Real Linux VPS server security is defense in depth: several independent layers, so that when one fails, another still stands. In this guide, those layers are:

• Access control: who can log in, and how
• Network controls: what can reach the server
• Patching: closing known holes
• Attack surface reduction: running less software
• Resilience: recovering when something goes wrong

Why Is Linux Server Security Important?

Linux is widely used for servers because of its flexibility, stability, and built-in security features. However, those features work best when your server is configured and maintained properly.

An unsecured server can expose sensitive information, interrupt website availability, or allow attackers to misuse your resources.

Some common risks include:

• Unauthorized access: Attackers may try to guess passwords or exploit weak authentication.
• Outdated software: Unpatched vulnerabilities can allow attackers to compromise your system.
• Open ports: Unnecessary services can create additional entry points.
• Application vulnerabilities: Outdated plugins, insecure websites, and poorly configured databases can expose data.
• Data loss: Malware, accidental deletion, or system failures can affect important files.

Securing your Linux server isn’t a one-time task. It requires regular updates, careful access management, and ongoing monitoring.

Before you proceed with securing a Linux Server

• You need root or sudo access and an SSH connection.
• Keep your current SSH session open while you change SSH or firewall settings. Test in a second terminal before closing the first. Most self-inflicted lockouts happen at steps 3 and 4.
• Know whether your provider offers a web console. If you do get locked out, that is your way back in.

Step-by-Step Guide to Secure a Linux Server

1. Update Your Linux Server Regularly

One of the easiest ways to improve server security is to keep your operating system and installed software up to date. Updates often include fixes for known vulnerabilities that attackers may already know about.

For Ubuntu or Debian run:sudo apt update && sudo apt upgrade -y

For AlmaLinux or Rocky Linux, use:sudo dnf upgrade

Review the proposed changes before confirming updates, particularly on production servers. Some updates may require a service restart or system reboot.

It’s also a good idea to establish a regular patching schedule rather than waiting until a security issue appears.

2. Create a Separate Non-Root Administrative User

Using the root account for everyday tasks gives commands unrestricted system-level privileges. A small mistake can affect the entire server.

Instead, create a separate administrative account:sudo adduser adminuser<br>On Ubuntu or Debian, Grant the user administrative privileges with:sudo usermod -aG sudo adminuser

Instead, create a separate administrative account:sudo adduser adminuser<br>On AlmaLinux or Rocky Linux, use the wheel group where sudo access is configured:sudo usermod -aG wheel adminuser

Log in with the new account and confirm that you can perform administrative tasks before restricting root access. Only trusted users should receive these privileges.

3. Set Up SSH Key Authentication

From your local machine:

ssh-keygen -t ed25519 -C "you@example.com"<br>ssh-copy-id deploy@YOUR_SERVER_IP

Then confirm that ssh deploy@YOUR_SERVER_IP logs you in without a password prompt (you may be asked for the key’s passphrase, which is what you want).

Why it matters: SSH keys can’t be guessed by brute force the way passwords can. Protect your private key with a passphrase.

4. Harden SSH & Configuration

Rather than editing the main sshd_config, use a drop-in file. Name it so it loads first, because OpenSSH uses the first value it finds for each setting:

sudo nano /etc/ssh/sshd_config.d/00-hardening.conf

PermitRootLogin no
PasswordAuthentication no
KbdInteractiveAuthentication no
MaxAuthTries 3

Check the syntax, then confirm the settings that will actually apply:

sudo sshd -tsudo sshd -T | grep -Ei 'permitrootlogin|passwordauthentication'PermitRootLogin no<br>PasswordAuthentication no

If sshd -T still shows passwordauthentication yes, another file is winning. On many cloud images, /etc/ssh/sshd_config.d/50-cloud-init.conf sets it. Fix that file or make sure yours sorts earlier.

Reload SSH (the service is called ssh on Ubuntu/Debian and sshd on RHEL-family systems):

sudo systemctl reload ssh

Now, in a new terminal, confirm you can still log in as deploy with your key before closing the old session.

5. Secure SSH Remote Access

SSH allows you to manage your Linux server remotely. Since it provides direct access to your system, protecting it should be a priority.

Use SSH keys instead of relying on passwords

SSH keys provide an alternative to password-based authentication. Generate an Ed25519 key pair on your local computer, if supported:

ssh-keygen -t ed25519

Copy the public key to your server:

ssh-copy-id adminuser@your_server_ip

Replace adminuser and your_server_ip with your actual username and server address. Keep your private key secure and never share it.

Disable direct root login

After confirming that your administrative account works, back up the SSH configuration:

sudo cp /etc/ssh/sshd_config /etc/ssh/sshd_config.bak

Edit the configuration file and set the following directives:

PermitRootLogin no
PubkeyAuthentication yes
PasswordAuthentication no

Validate the configuration before reloading the SSH service:

sudo sshd -t

If the check succeeds, reload the appropriate service for your distribution, commonly ssh or sshd.

Changing the default SSH port may reduce automated scanning noise, but it doesn’t replace proper authentication and access controls.

6. Configure and Enable a Firewall

A Linux firewall configuration helps control which network connections can reach your server. Ideally, only the ports required by your applications should be accessible.

On Ubuntu or Debian systems using UFW, first allow the SSH connection you currently use:

sudo ufw allow OpenSSH

If your server hosts a public website, allow HTTP and HTTPS traffic:

sudo ufw allow 80/tcpsudo ufw allow 443/tcp

Set the default traffic policies:

sudo ufw default deny incomingsudo ufw default allow outgoing

Enable the firewall and check its status:

sudo ufw enablesudo ufw status verbose

Before enabling it, confirm that the SSH rule matches your actual SSH port. If you’re using a custom port, allow that port instead.

On AlmaLinux or Rocky Linux, firewalld is commonly used instead. Review its active zone and existing rules before making changes.

You should also check for unnecessary listening ports:

sudo ss -tulpn

Avoid exposing databases, internal APIs, and administrative interfaces directly to the public internet unless there’s a clear need and appropriate access restrictions.

7. Turn on Automatic Security Updates

Most real-world compromises exploit vulnerabilities that already have patches. Automating security updates closes that window:

sudo apt install unattended-upgradessudo dpkg-reconfigure --priority=low unattended-upgrades

By default this applies security updates only, not every package, which keeps the risk of breaking changes low. Kernel updates still need a reboot to take effect; check for /var/run/reboot-required or configure an automatic reboot window if your site can tolerate it.

8. Block Repeated Login Attempts with Fail2ban

Automated bots often try to access servers by repeatedly guessing passwords. Fail2Ban can help by detecting repeated failed attempts in supported logs and temporarily blocking offending IP addresses.

On Ubuntu or Debian, install and enable it with:

sudo apt install fail2bansudo systemctl enable --now fail2ban

Check whether the service is running:

sudo fail2ban-client status

Review the SSH protection status and adjust retry limits and ban duration to suit your environment.

Fail2Ban is an additional layer of protection, not a replacement for SSH keys, firewall rules, or regular updates.

9. Disable or Restrict Unused Services

Every service running on your server can potentially introduce another security risk. If you don’t need a service, you should consider disabling it after confirming that it’s not required by your applications or operating system.

List running services:

sudo systemctl --type=service --state=running

Review listening ports:

sudo ss -tulpn

Investigate unfamiliar services before changing anything. Disabling a service without understanding its purpose could interrupt your website, database, or other applications.

10. Protect the Web Layer

If the server hosts websites, the application is usually the weakest point, not SSH. Priorities:

• Keep the CMS, plugins, and themes updated. For WordPress
• Run a supported PHP version.
• Consider a WAF (Web Application Firewall), which inspects HTTP requests for patterns such as SQL injection and cross-site scripting. A WAF reduces risk but does not fix a vulnerable plugin; it buys you time.
• Run each site under its own system user (separate PHP-FPM pools) so one compromised site can’t freely read another.

11. Use HTTPS Everywhere

On Ubuntu or Debian systems using Certbot, obtain and configure an SSL/TLS certificate for Nginx with:

sudo apt install certbot python3-certbot-nginxsudo certbot --nginx -d example.com -d www.example.com

Install a TLS certificate and redirect HTTP to HTTPS. Free certificates from Let’s Encrypt are the standard choice, and certificate lifetimes are getting shorter, which makes automated renewal mandatory.

12. Apply Least-Privilege Access to Files and Databases

Securing the operating system is only part of the job. The applications running on your server need attention too.

Follow these practices to reduce common vulnerabilities:

• Keep website software, plugins, themes, frameworks, and dependencies updated.
• Use strong, unique database credentials.
• Give database users only the permissions they actually need.
• Restrict database access to trusted hosts or private networks.
• Install valid SSL/TLS certificates and configure HTTPS.
• Keep API keys, private keys, and environment files away from public repositories and web-accessible directories.
• Remove unused applications and outdated components.

These steps are especially important for servers hosting customer websites, business applications, or sensitive information.

13. Set Up Reliable Backups and Test the Restore

Even with strong security controls, unexpected problems can happen. A reliable backup gives you a way to recover from accidental deletion, hardware failure, or a security incident.

Make sure your backups include important website files, databases, configuration files, and other critical data.

For better protection:

• Automate backups and check that they complete successfully.
• Keep an independent or off-site copy.
• Encrypt sensitive backup data where appropriate.
• Set a retention policy so you have access to earlier versions.
• Test restoration regularly.

A backup isn’t truly reliable until you’ve confirmed that you can restore it successfully.

If you suspect your server has been compromised, isolate it where practical, preserve relevant logs, revoke exposed credentials, and investigate the cause. Depending on the incident, restoring from a known-clean backup or rebuilding the server may be safer than simply deleting suspicious files.

14. Monitor Logs for Suspicious Activity

Regular server monitoring helps you spot unusual activity before it develops into a larger problem.

To review recent login history, run:

last

On systems that use systemd, review warning-level and higher-priority journal entries with:

sudo journalctl -p warning

Authentication logs vary by distribution and logging configuration, so check the relevant logs for failed login attempts and unexpected account changes.

It’s also useful to monitor CPU usage, memory, disk space, network activity, and service availability. For production servers, configure alerts for unusual activity, service failures, and backup problems.

Malware scanners such as ClamAV can help identify known malicious files, but a clean scan doesn’t guarantee that a server is free from compromise. Investigate unexpected processes, scheduled tasks, and unexplained resource usage.

How to Verify Your Linux Server Hardening

Run through this quick audit:

You can also scan your own server’s public IP from another machine with nmap to see what an outsider sees. Only scan servers you own.

Linux Server Security Checklist

Before considering your initial security setup complete, review the following checklist:

• The operating system and packages are up to date.
• A separate administrative user is configured.
• SSH key authentication has been tested.
• Direct root login is disabled where appropriate.
• Password-based SSH authentication is restricted where appropriate.
• Firewall rules reviewed and unnecessary ports closed.
• Unnecessary services and ports are disabled or restricted.
• Brute-force protection configured.
• Unnecessary services disabled.
• Websites, databases, and TLS settings are secured.
• Backups are automated, and restoration is tested.
• Logs, alerts, and suspicious activity are monitored.
• Incident recovery plan documented.

Common Linux Server Security Mistakes to Avoid

Conclusion: Keep Your Linux Server Secure

Learning how to secure a Linux server starts with a few essential practices: regular updates, restricted access, SSH hardening, firewall configuration, application security, backups, and monitoring.

You don’t need to implement everything at once, but you should make security part of your regular server maintenance routine. Test configuration changes carefully, keep a recovery plan, and review your setup whenever your infrastructure changes.

Simplify Your Server Management with HostsNiX. Managing server security takes ongoing effort. Explore HostsNiX to discover server management tools designed to help organize and simplify your hosting operations as your infrastructure grows.