Unlocking the Cloud: Exploring Distributed Password Cracking and the Power of Hashtopolis
12. 8. 2023 | 10 min read | author: Jan Ulrich
As technology continues to propel us towards a cloud-centric future, the question of password security looms ever larger. Can the formidable computing power of GPU cloud instances be harnessed for password cracking?
The article's primary objective is to explore the potential of cloud GPU providers and investigate the well-known solution for distributed password cracking, Hashtopolis. The focus is to provide a straightforward solution without developing custom tools or reinventing the wheel. Instead, we intend to leverage the functional solutions already available.
Jan Ulrich is an Offensive Security Specialist passionate about malware development, phishing and password cracking. After graduating from the Czech University CVUT as a Structural Engineer, he soon discovered that this path was not fulfilling for him and decided to pursue a career in information security. In a great effort, he finally got a chance at axelum, where he has been working successfully for over two years, learning from more experienced colleagues and hoping that his actions will contribute to greater digital world security.
I embarked on an investigation to reveal the potential of distributed password cracking in the cloud. During this exciting investigation, a surprising revelation emerges in the form of Hashtopolis. This clever solution not only leverages the vast resources of multiple cloud-based GPU instances simultaneously but also automates and streamlines monotonous and repetitive attacks. As traditional hardware collides with cloud GPUs, these findings change our understanding of password security.
My motivation for writing this article arose during our team's recent Active Directory assessment, which I had the privilege of participating in. Several times, we found ourselves in situations where we were able to obtain hashes, the cracking of which was all that stood out between us and compromising the entire Active Directory, including the domain admin account. At this point, I realized the importance of cracking passwords during the Active Directory assessment or Red Team operations.
Beyond Hardware Limits
Despite having hardware with decent computing power at our disposal, what more could be done to increase our chances of success? Of course, an intelligent person would first focus on things they can easily improve, such as better word lists and rules. However, there are limits to what can be achieved through such means, and very quickly, the limiting factor becomes the performance of your password-cracking rig. I discovered that renting the most powerful NVIDIA A100 80GB graphics card worth around $23,000 is now possible for a mere $1.1 per hour on LambdaLabs. Obtaining an instance of this graphic card, installing Hashcat, and running a few benchmark tests took only an hour.
The gained performance was comparable to an average password-cracking rig (Hashcat benchmark with a result of 100 NTLM GH/s). However, I have experience with Terraform and Ansible, powerful tools for managing and configuring infrastructure by typing a single command into the terminal. This led me to the idea of distributed password cracking in the cloud, achieved by distributing individual cracking subtasks to different GPU instances at a fraction of the cost of purchasing and maintaining hardware with equivalent performance.
Already Existing Tools
You may be wondering if someone has already come up with the idea of cracking passwords in the cloud, and you would be right. The Coalfire Labs R&D group developed one solution, but unfortunately, the organization no longer maintains it. However, an individual has taken over the project and continues to support it.
The NPK is a tool that uses Hashcat for the actual password cracking, with a strong emphasis on ensuring that the allocated budget for cracking is not exceeded. However, its most significant advantage is its automated deployment of GPU instances, unfortunately, is limited only to the AWS cloud provider. The solution itself is a black box to me. When starting a cracking campaign, you click through the attack configuration in the GUI, specify the number of GPU instances or the duration for which the GPU instances should run to allow Hashcat to iterate through the entire keyspace, and then run the attack. I miss the view under the hood, how exactly it works, what exact Hashcat command was executed on each instance (was the-O switch used to optimize the kernel, which in most cases significantly increases the cracking performance), what is the usage and temperature of each GPU, and other detailed information. The solution is okay but lacks the flexibility and transparency a security specialist needs when cracking passwords.
Cloud Providers
During an internet search, I found a list of the most well-known cloud service providers offering GPU instances, such as Amazon Web Services (AWS), Microsoft Azure, Baidu AI Cloud, Google Cloud Platform (GCP) and more.
- Microsoft Azure - 1x A100 3.673$/hour
- Google Cloud (GCP) - 1x A100 $3.930/hour
- Vultr - 1x A100 $2.604/hour
- LambdaLabs - 1x A100 $1.100/hour
- Amazon Web Services (AWS) - 8x A100 - $40.945 - This contrasts our finding on the parallel use of GPUs at the instance level.
Setup
For distributed password cracking in the cloud, I need to create an infrastructure consisting of one control Server and one or more Agents that will perform the heavy cracking work.
1. Server
I chose one of the smaller VPS at the cloud provider Hetzner, for the server, with 2 CPU cores, 4GB of RAM, and 40GB of disk space. A detailed description of the server installation process can be found in the Hashtopolis server installation guide. It involves installing a few system packages, creating a database in MySQL server, and downloading the Hashtopolis project from GitHub. The installation takes no more than an hour, and the entire deployment and server installation process can easily be automated with Terraform and Ansible.
2. Agents
For agents, I chose a mix of LambdaLabs A100 and A10 GPU instances.
Once the GPU instance is ready, we can add its IP address to our Ansible Inventory and let Ansible complete the installation. The agent installation process is straightforward. We only need to install the psutil and requests python3 packages using pip3 and download our server's automatically generated hashtopolis.zip file. This file is then executed with the required parameters using the command pythos3 ./hashtopolis.zip --url \\url of our hashtopolis server\> --voucher generated in hashtopolis server\>, and the agent is paired with the server.
The final state looks as follows: In the server's web interface, you can see the connected agents.
On agents, we receive information about what the agent is currently doing, for example, whether it is currently cracking an assigned chunk, downloading a wordlist or waiting for a task.
Testing
I aimed to test the real-world use, so I will not show you just a few numbers from the Hashcat benchmark. Instead, I will present the advantages and disadvantages during, for example, a red team operation when you have already compromised a host in Active Directory and obtained 5 NTLM hashes. Cracking them would give you further possibilities to compromise Active Directory.
Hashtopolis vs Hashcat
Before the final comparison of the performance of each configuration, I would like to highlight the main difference between working with Hashtopolis (server + agents) and using Hashcat on a single host. When using Hashcat on a single host, you always enter one task at a time and wait for it to finish. However, with the Hashtopolis server, you can define multiple tasks prioritized and processed sequentially by the agents.
This goes even further, as you can define so-called pretasks, which are configurations of attacks themself without specifying a hashlist. It makes sense to configure the most typical attacks in this way, which are always run during the cracking process. For example, a brute-force attack up to a certain number of characters or popular wordlist attacks.
You can then combine these individual pretasks into so-called supertasks, a group of smaller tasks. Subsequently, for the created supertask is defined a hashlist, which will be cracked, and the supertask is added to the regular queue of all tasks.
At first glance, this complex structure is convenient. Not only can you easily define a queue of attacks to be executed, thus ensuring the maximum utilization of all agents, but you can also greatly simplify repetitive attacks.
Benchmark
Now let's get to the actual performance comparison. We will compare the performance of three setups:
- Dedicated host with 3x Nvidia RTX 2080 Ti
- Hashtopolis server + 3x Nvidia A100 & 3x Nvidia A10 (6 agents)
- Hashtopolis server + 10x Nvidia A10 (10 agents)
Setup | Price [$/hour] | Bruteforce attack [GH/s] | Wordlist + rules attack [GH/s] | Bruteforce attack performance/price [GH/$] | ||||||
| 3x RTX 2080 Ti | - | 100.50 | 44.75 | - | ||||||
| 3x A100& 3x A10 | 5.1 | 279.85 | 105.88 | 197,541 | ||||||
| 10x A10 | 6.0 | 340.48 | 135.12 | 204,288 | ||||||
The last column of the table above is a calculated pseudo-value that indicates how many billions of hashes (giga hashes) we can compute for one dollar during a brute-force attack. This quantity is only used for comparing the cost-effectiveness of each instance. From the table, despite being quite different, our two cloud instances achieve very similar cost-effectiveness.
What is also very important is that the bruteforce attack of a single A10 (tested separately)achieves a performance of 33.85 GH/s. This means the Hashtopolis solution does not lose performance with the number of agents and scales linearly.
Comparing Dedicated Hardware and Cloud GPU
So which option is more beneficial for password cracking, dedicated hardware or GPU cloud instances? An easy answer does not exist.
According to the recorded measurements in the table, utilising distributed password cracking in a cloud environment demonstrates consistent cost and the potential for significant performance. Consequently, this approach represents a versatile solution tailored to the user's specific requirements. However, reliance on cloud providers and the availability of GPU instances is a significant concern. On the other hand, maintenance of dedicated hardware is also time and cost-intensive. The purchase cost of three Nvidia RTX 2080 Ti cards is $3000, which could cover 1500 hours (62.5 days) of cracking in the cloud with equivalent performance.
Empowering Password Cracking
My brief research has confirmed that distributed password cracking in the cloud is feasible and cost-effective, thanks to the availability of existing solutions. However, my findings have led to another unexpected conclusion. I recommend using Hashtopolis, even on dedicated local hardware, due to its ability to automate tedious tasks and enable the temporary combination of dedicated local hardware with cloud GPU instances to increase performance when needed.
Read More
Many people have a romanticised view from the films that an attacker is going to immediately pick the locks on a door in order to gain access in the depths of the night to steal sensitive information. Does it really work that way? Physical security is still one of the most overlooked and underfunded areas for major organisations.
Right in Your Inbox
Stay up to date and get the newsletter. Every month, you can look forward to exclusive educational content and news from the infosec world.
