Penultimate weekend, we hosted our very first jeopardy style capture the flag event: The Internetwache CTF 2016
In this blogpost, we will write about the CTF from the organizer’s perspective. What was the setup? What went wrong? What did we learn? What was good? What can we do better next year? We hope that this insight can help other CTF organizers in the future.
First of all, some words about us: Sebastian (aka gehaxelt ) participated as part of the ENOFLAG team in various CTFs (jeopardy / attack-defense) in the past. Tim (aka TPS) is not a regular CTF player - but was curious about helping Sebastian to host the first Internetwache CTF.
Somewhen during last november, Sebastian had the idea of hosting a jeopardy-style CTF. He was curious to see what it takes to host a CTF and to create challenges for other hackers. Fast forward to the beginning of february 2016 - We announced the CTF on ctftime.org and got an initial rating weight of 5.00. Afaik the rating is based on the ctftime-admin’s subjectivity. As we assumed a weight of 0.00 (like other CTFs), we were ok with that, but hope to get a better rating next year :)
The CTF should take place on the the 20th of february 2016, so there were around 3 weeks to setup and finalize everything. A ‘speciality’ was, if you want to call it that, that we did not sort the challenges by difficulty. There were two reasons for this: First is that it was a bit hard for us to estimate the difficulty of a challenge. Second is that you should be able to find the tasks you can solve in a short period of time - similarly to exams ;)
All in all we think that the CTF wasn’t bad - at least we had a lot of fun and positive feedback from the community. We’ll have a look at the feedback later on.
Some numbers:
Simply to give you an overview over the CTF in numbers:
- Scoreboard: https://ctf.internetwache.org/scoreboard
- Duration: 36 hours (20 Feb. 2016, 12:00 CET — 22 Feb. 2016, 00:00 CET)
- Registered teams: ~1500
- Active teams: ~650
- Teams that solved all challenges: 38
- Ctftime.org rating: 4.5
- Ctftime.org weight: 5.00
- Wall of Shame: 35 IP addresses
- HTTP requests: 2336957
- Traffic: ~20 GB
- Costs: 20$ (Hosting)
- Prizes: 0 (None) (If you want to be the sponsor for next year’s edition, feel free to send us an email :) )
We were excited to have as many as 650 active teams (solving at least one challenge). That’s a bit less than 50% of the total registration count. What disasspointed us were the 35 people who didn’t read the rules and used agressive automated tools and won a place in our iptables ruleset. However, some apologized and we were happy to unblock them again :) Another number that we over-estimated was the amount of traffic or resources in general.
Setup
For the hosting part we relied on Digitalocean.com, because we still had some bugbounty reward $$$ on our account and Sebastian is satisfied with them. Further reasons were that spinning up VMs takes less than a minute and that snapshots are free of charge what made testing different configurations a great experience.
We rent 4 VMs in total:
- 1x $5/mo as the monitoring VM, which collected and displayed the performance data of all other VMs with Collectd
- 3x $80/mo VM for the proxy, service and webserver VMs.
The ctf.internetwache.org domain pointed to a floating-ip which in turn pointed to the proxy-VM. All four VMs were interconnected using Digitalocean’s private network feature.
On the proxy-VM we configured NGINX to act as a load-balancer for the HTTP(s) and the TCP traffic.
The web traffic was proxied to the webserver VM and the TCP traffic to the service VM. This central proxy allowed us to easily stop malicious attackers and scale to more VMs if needed.
The webserver VM ran apache2, mysql and php5. We used the apache2-mpm-itk module to assign every vhost a different user. The service VM used tools like Daemontools and TCPServer to host the services.
All VMs had Cgroups configured to limit resources of the CTF users in the ctf group.
This setup seems to be pretty solid, because we didn’t have any major load or stability issues during the CTF. In retrospect we could have used smaller VMs, but the cost difference was only ~$6.5 for the whole event and that was totally worth not having to worry about resources. During the CTF we only faced one minor incident.
We based our scoreboard on the tinyctf-platform. It’s written in python, easy to extend (we implemented CSRF protection and other features) and nice to look at. This scoreboard is Internetwache-approved and we may use it again next year.
Challenges
Let’s have a look at the challenges. There were six categories with five challenges each:
Misc
- Misc50: Octal and base64 encoding
- Misc60: Base64 and QR codes
- Misc70: Pcap dump with zip file
- Misc80: DNS requests and hex
- Misc90: Barcodes
Web
- Web50: PHP magic hashes
- Web60: PHP preg_replace with e modifier RCE
- Web70: MySQL truncation vulnerability
- Web80: Public git directory
- Web90: Latex RCE
Rev
- Rev50: MIPS assembly
- Rev60: File content checks
- Rev70: Switch case input checks
- Rev80: TapeBagel reversing
- Rev90: Rubiks Cube with flag
Crypto
- Crypto50: Multiple ciphers chained
- Crypto60: RSA key factoring
- Crypto70: Hash collisions
- Crypto80: Stegano / DTMF
- Crypto90: Modify ciphertext
Code
- Code50: Solve equations
- Code60: Find prime numbers
- Code70: Solve encoded equations
- Code80: Bruteforce a string
- Code90: BST tree operations
Exploit
- Exp50: Ruby Regex
- Exp60: Integer-Overflow
- Exp70: Variable-Overflow
- Exp80: Formatstring vulnerability
- Exp90: NodeJS ‘shell’
You can find all challenges and some configuration files on our GitHub repository.
Easter-Egg: All port numbers were primes ;)
Problems we faced
We are all humans and we also did some mistakes or faced some problems during the CTF. We think that it is important to talk about these issues:
Web70
The intended solution for this task was to use a mysql truncation vulnerability. This vulnerability worked way too good and once successfully exploited, allowed other teams to login with admin/admin or similar combinations. This obviously made the challenge way too trivial. We took down web70 for a while and implemented a quick ‘n’ dirty fix.
Web90
During the creation of the challenge, Sebastian thought of a specific way to solve it. Unfortunately, he forgot to disable/filter the straight forward \write18 command, what made this challenge trivial to solve with \immediate\write18{command}. However, we noticed this bug way too late to hot-patch it. We hope that you still learned something new :)
Sebastian is going to write a blogpost on 0day.work about the intended solution eventually.
Rev90
This was the Rubik’s Cube challenge. Sebastian did a small mistake while manually scrambling the cube. A corrected version of the README was uploaded during the CTF. Sorry for that! (However, there was another way to solve the challenge without a cube)
Crypto50
This seemed to be an easy challenge from our point of view, but turned out to be way too difficulty (too much guessing?) for the participants. We should have made the description more clear.
Code70
The time format (TIME:CHAR) was a bit unclear.
Code90
The description was a bit unclear about the expected input/output format.
Duration
The duration was another thing that we should have chosen a lot shorter. We had the feeling that after around 24 hours there wasn’t much activity anymore. Only some teams who wanted to solve the remaining couple of challenges. There was, however, a peak during the last few hours again. People told us that longer CTFs are good for newcomer teams and timezone-daylight-differences, so in the end it wasn’t a problem at all. The next CTF will definitely be shorter (around 12 hours probably). Sebastian learned that in order to stay awake for 40 hours straight, two bottles of Club Mate are enough ;)
Difficulty
As this was the first time we hosted a CTF, it was hard for us to estimate the difficulty of the tasks. It turned out that the tasks were a bit too easy. However, as said above this helped newcomers to learn to ‘how to CTF’. We’ll try harder next time!
Less hints
We received the critique that we gave away too many hints via private chats. In retrospect that may be correct, but we tried to keep people motivated by asking them questions like ‘Where are you stuck? What did you try? What else do you know? What can you think of?’ and let them figure out the rest. In our opinion that reduces frustration and raises knowledge. But next time we’ll try to keep the hints more general and public (a separate ‘hints’ page is a good idea).
All in all, most of these problems could have been avoided with more thorough testing in the preparation phase.
Writeable directories
Some people told us that they solved a challenge by grepping for the flag format on the whole filesystem. That lead to some trivial flags because some files in directories like /tmp/ contained them. We fixed this issue during the CTF by using ACLs to remove write-permissions for users of the ctf group to directories like /tmp, /var/tmp and so on.
Load spikes
There was one interesting incident. We still don’t really now what happened, but the load spiked to 2600 and the RAM to 6gb at the same time. It looked like the Cgroups 6gb RAM limit was hit. Interestingly the VM was responsive and a restart of apache resolved the problem.
What we think we did well
Okay, let’s talk about some positive aspects of our CTF. In our humble opinion the uptime and overall stability of the services was good. Except the minor service downtimes due to bugfixes, all services were reachable during the CTF.
Another strong point was the communication. We were available on twitter and IRC throughout the whole CTF. We did not actively observe cheating, flag sharing or other bad behaviour on our IRC channel.
Giving some people an ‘ohh’-effect while solving the challenges. It seems like we allowed newcomers to enjoy and learn about CTFs and the good feeling when you finally solve a challenge.
Some refreshing and/or interesting challenges like exp80 / exp90 / web90 / rev90 / rev80 (Sebastian’s personal favorites).
What others think about the CTF
We used a Google Form to ask for feedback. 80 people were so kind to answer a handful of questions and we would like to share some answers with you.
Some charts:
General challenge feedback:
- Challenge and scoreboard availability was good, services seemed to be mostly up and responsive for me, even with so many participants, nice job on that! However, we had some beginners on the team and I think it’s great to have easy challenges as well so they don’t get frustrated. They should not constitute the heart of the contest though.
- Very good, especially for the 1st CTF you’ve run!
- A bit on the easy side - but that is also fine sometimes. Great quality in presentation, and descriptions.
- It seems little bit easy to solve the challenges, but even if that we’ve learned lots of new stuff from challenges in time.
- I liked that the difficulty was not immediately visible. Encouraged me to try all the challenges and not to start with the easy ones.
Final words:
- Extremly well done for the first time hosting a CTF, and one of the most enjoyable CTFs so far this year. Good challenge difficulty for beginning CTF teams. Time limit was okay, bit long maybe, but that leaves the less active teams time to enjoy the challenges
- Great IRC moderators! This CTF was well run which is very much appreciated after several badly run CTFs this year.
- Good job.Thanks :)
- Awesome job guys, looking forward to next year!
- I’d like to point out that your organizing work is clearly above average.
- Unlike some CTFs, this has potential to not suck. It’s a little rough right now, but I think everyone forgives you for minor mistakes in the beginnings of a major undertaking.
We are looking forward to hosting another CTF (presumably) next year!
The team of internetwache.org