Are you looking to set up your own SMTP server but don’t know where to start? Look no further! In this guide, we’ll walk you through the step-by-step process of creating an SMTP server on a VPS using the Hestia control panel.

With this SMTP server, you’ll have access to a reliable and secure platform for hosting your email server. And with the Hestia control panel, configuring and managing your SMTP server is a breeze.

By the end of this guide, you’ll have your own fully-functioning SMTP server up and running. So, whether you’re a small business owner or an individual looking for a personal email server, this guide is for you.

Ok, let’s start the journey.

Creating an Instance on your Server Provider

The first thing you need to do is create a server to host this SMTP server. You can use any server provider for that. Check the below link, if you want to find a cheap server provider.

I am using the Oracle Cloud free tier to get a free server.

Once you have signed up for your server provider account, the next step is to create an instance. An instance is a virtual machine that serves as the foundation for running your applications and services on the provider you choose.

I am going to create an instance on Oracle Cloud, so you can follow the steps to create a server on your preferred server provider.

You have to select an operating system that supports the Hestia control panel.

I selected Ubuntu 20.04 LTS as the operating system for this tutorial. You can choose either AMD or ARM64 architecture to host your SMTP server; Hestia Panel supports both.

Once the instance is created, you will be provided with a unique public IP address and other information that you will need to access your instance, such as SSH credentials.

Install Hestia Control Panel on VPS

Installing Hestia Control Panel on your deployed server is the next step. The Hestia Control Panel is a web-based interface that allows you to easily manage and configure your VPS.

To install the Hestia Control Panel, you will first need to log in to your VPS via SSH. You can use any SSH client for this, such as Putty SSH or Bitvise. Now I am going to connect to my server using my SSH key. Ok, now I’m in.

Once logged in, run the following command to get root access:

sudo -i

Then run this command to update all the applications on your Linux server before the actual installation.

sudo apt update

Next, run the following command to download the Hestia installer script:

wget https://raw.githubusercontent.com/hestiacp/hestiacp/release/install/hst-install.sh

Then, execute the script by running this command:

bash hst-install.sh -f

After running the script, you will be prompted to provide some information, such as your hostname and administrator email.

After running the script, you will be prompted to provide some information, such as your hostname and administrator email.

Enter a valid email address here. Next, it asks you to submit a hostname for your server.

It is better if you can use a hostname with a top-level domain name. It should be a subdomain; you can’t add the root domain as the hostname.

You can take a brief break while the entire installation is being completed; it takes a few minutes. I will speed up the video.

Ok, now the installation is complete. At the end of the installation, it displayed the admin username and the password for Hestia Panel; copy them into a notepad.

Open Firewall Ports on your Server

In order for your Hestia Panel SMTP server to function properly, certain ports need to be open on your server’s firewall.

Specifically, ports 25, 465, and 587 need to be open for incoming and outgoing SMTP traffic. You should also open ports 8083, 443, and 80 for the Hestia Panel web interface.

25, 465, 587, 8083, 80, 443, 993, 143, 995, 110

This process may be slightly different depending on the server provider, but you can follow the same sequence.

Go to the firewall section of your server provider; usually, it is on the security section of the server. Then you have to add a new firewall rule to open these ports.

0.0.0.0/0

Use this as the source IP; it allows all IP ranges to connect to your server.

Then enter your ports, which need to be opened as destination ports. Because I already have other ports open, I’m only opening these.

Configure Hestia Panel Web Interface

After opening all the necessary ports, you can enter the Hestia web interface. You can access it by visiting the IP address of your VPS and the 8083 port in a web browser and logging in with the administrator credentials you copied after the installation process.

https://ipaddress:8083

When you enter it for the first time, you will see this warning sign due to the absence of an SSL certificate. Click the “Advanced” option to proceed.

Enter your login credentials here. Ok, now you are in the Hestia Control Panel. Next, we have to add a new user before adding the domain name for the SMTP server.

Click the “Add User” button, and you will be directed to this screen. Enter a username and the preferred nickname.

You can use a generated password as well. It is better to note down these credentials. Change the “Role” to “Administrator.” After that, click on the “Save” button in the upper right corner.

Ok, now you have successfully added a new administrator user to the Hestia panel.

Add a New Domain to Hestia Panel

You should add a top-level domain name to the Hestia panel to create an SMTP server. It is critical to include a top-level domain such as .com ,.net, or .org to prevent emails from being routed to spam folders.

If you want to buy a top-level domain name, check the below link.

You can also use a free domain, but it is not suitable for a standard SMTP mail server.

After buying a domain, you should add a DNS record to point out the server’s IP. Go to your domain registrar and navigate to the DNS section.

Add an “A” record to the root domain or a subdomain and enter the server’s IP address as the destination. I will use the main domain, so I enter the “@” symbol as the name.

Then go to the Hestia Panel and switch to the newly created user. Under the “Users” tab, you can choose your non-system admin account and click on the small login icon. We have the login to the new user now.

Click the “add web domain” button to add the new domain. Then enter your domain name here and check the “Enable mail for this domain” box. You can click the “save” button to add the domain if everything is correct.

It has been successfully added to the system. Let’s configure the SMTP server settings.

Add SMTP Mail Server Certificate to DNS

Once your SMTP mail server is set up and configured, it’s important to add a certificate to your DNS records in order to ensure that your emails are delivered securely.

Hestia Panel creates all of the necessary mail records for your SMTP server automatically. So you just need to add it to your DNS records for your domain.

To check your SMTP DNS records, click on this small icon in the DNS section. Ok, these are your mail server DNS records. You have to add these ones by one to the DNS section of your domain.

You can add the certificate to your DNS records by logging in to your domain registrar’s account and navigate to the DNS management section.

Here, you will be able to add a new record for your domain. Follow the same record data in the Hestia panel. You just need to copy and paste the records.

The first two are “A” records; copy the name and IP address and paste them into your domain name provider’s DNS record. The third one is a MX record; you have to select the MX record in the DNS section and enter name, mail server, and priority.

Others are TXT records; copy those values and names to the DNS. Ok, now you are finished with adding the mail certificate for your SMTP server.

By adding the certificate to your DNS, you will be able to encrypt the email communication and protect your email from being intercepted by unauthorized parties.

Add Mail User Accounts

After navigating to the “Mail” section of the Hestia panel. Then click the “plus” icon to add a new mail user to your SMTP server. In this window, enter a name for your account.

The name you entered here is your email address for this mail server. You can use it to send and receive emails. You can either enter a strong password or use one generated here.

Copy these credentials to the notepad. These ports, username, password, and mail server address are useful to send email and integrate with some applications.

To add this user to the server, click the “save” button after that.

Testing the SMTP Server

After setting up and configuring your SMTP server, it’s important to test it to ensure that it is working properly. To test the server, you can send an email to yourself or another recipient.

I’m doing this with two web tools: the Gmass SMTP mail sender and the mail tester to check the mail score of receiving emails. This will allow you to check the communication between the client and the server and troubleshoot any issues that may arise.

The mail score is a measure of the overall quality and deliverability of an email. It takes into account a variety of factors, such as the sender’s reputation, the domain, the email client, and the recipient’s inbox.

A high mail score indicates that your email is likely to be delivered to the recipient’s inbox, while a low score may result in your email being blocked or sent to the spam folder. Ok let’s check you SMTP server.

When you visit the mail tester website, you will see a temporary email like this, which you should copy and paste into Gmass sender as the “To email”. You have to fill in this information form with previously copied mail user credentials.

Enter the mail server, port number, username, password, and email address. As the port number, you can use the default SMTP port 25, or if it is blocked, use another port like 465.

Next, you have to click the “Test it” button to send the email. You can see our server sent the email without giving any error messages. Now go to the Mail Tester and click the “Check your score” button.

You will receive your mail score, and it will also give you information about troubleshooting your SMTP server and increasing your mail score. Because you can’t set reverse DNS for our mail domain in the Oracle Cloud free tier, this SMTP server gets a mail score of around 6.

So it caused this huge drop in mail score; if you want 10 out of 10 score, use a premium VPS server. You can find cheap premium VPS providers in the article; check the description.

Check Spam Score of SMTP Server

The spam score is a measure of how likely an email is to be considered as spam by email filters and clients. A high spam score can result in your emails being blocked or sent to the recipient’s spam folder.

Let’s send a email to a gmail address and check whether it goes to inbox or spam box. Change the “To email” address to your gmail address and send a test email from the Gmass SMTP sender.

The test email will have arrived in the Gmail account’s inbox. This confirms that our SMTP server is up to standard.

Keeping the SMTP Server Updated

It is important to keep your SMTP server updated in order to ensure its security and stability. Additionally, keeping your server updated will also help improve its performance and reliability.

To update your SMTP server, you can use the built-in update feature of your control panel, or you can manually update the software by downloading the latest version and running the appropriate commands.

Bottom Line

In this guide, we’ve walked you through the process of setting up an SMTP server on a VPS using the Hestia Control Panel. We’ve covered everything from A to Z.

By following these steps, you should now have your own fully-functioning SMTP server up and running. We hope that this guide has been helpful and that you found it easy to follow.

If you have any further questions or need additional help, please feel free to open a thread on our forum.