Eh, my server was hacked
Last week, I got a email from Linode, which says my server is attacking other servers.
We have received a report of brute force attempts originating from your Linode. It appears that a process internal to your Linode is attacking other servers and attempting to guess their credentials.
Confused, I started to investigate the issue with accompanied guide.
If you believe that your Linode has been compromised, you can start troubleshooting by auditing the following log files and writable directories:
- /var/log/auth.log : Check this log file for signs of unauthorized access and brute-force attempts. Use the ‘last’ command to cross reference recent account logins with this file.
- /tmp : This directory is often used by malicious parties to store files
- Web server logs: There may be a vulnerable script or web application. The location of these log files depends on your web server (apache, nginx, etc.) configuration.
- ps aux : Use this command to audit running processes for foreign processes
In auth.log, I found a lof of failed logins(more than 17,000 times) from different IPs and different VPS provider that day. Someone was trying to hack my server! Just after I enabled password login several days ago. So I go to /etc/ssh/sshd_config to disable PasswordAuthentication so hackers can't do force attempts again. But I still didn't think it's compromised, anyway, I was using a strong password generated by 1Password, not any weak password, how could it be?
Then I spotted several suspicous processes such as /bin/bash 50 .go. It started again after a while if killed. What's the file of .go? Digging for several hours to find the source, I finnaly found the hacker's code. It's in /tmp directory, disguised as created by my Ghost blogging system. And it's mining Monero. Download the code here if you're interested.
By searching the file name of /tmp/.ssh/.rsync/c/lib/64/tsm in the hacker's code, I found sevearal related hacks: 1, 2. Relieved, I start to do things to secure my server. It's good to learn the importance of security when it's not too late.