Challenge writer Write Up: Security Fest 2018 CTF challs

Hi!

In this post I will note down my procedure for coming up with the challenges for SecurityFest CTF. The idea is explaining my side of the creative process in the hope that it can be useful to other people organizing CTFs. I will finish sharing some personal experience on the other stages of the process like difficulty and scoring.

Documentation process

As you might know this year’s CTF had as topic The Matrix. It had been a long time since I saw the movies (with Animatrix being the last one around 10 years ago) so I needed a refresher in the “lore” surrounding The Matrix.

I ordered a copy of the films and saw them one after the other with a notepad to note elements that could be interesting for a challenge and ideas that came up. I also spent some time reading articles from The Matrix wiki to get some references I may have missed and a taste of the extended lore.

With this I had some ideas on what challenges I could make and which parts of the history could allow for related cryptography and miscellaneous challenges. I also had some interesting ideas for keeping the IRC channel associated to the CTF interesting (either by using quotes or posting them on the topic).

Agent Communications

This was the first challenge I made. I was looking at symmetric ciphers common in the 90s (the age in which The Matrix Megacity is based) and decided to use some block cipher in ECB mode. Originally I was going to go for DES but I found a simple C implementation of blowfish which was perfect for this challenge. Although Blowfish is usually defined as having keys from 32 to 448 bits, this lower limit isn’t defined on the original paper by Schneier. In fact due to the cyclic nature of the key expansion process even a 1 bit key could be used.

Because of this I intended to make a simple challenge on which the encryption key could be easily bruteforced. Benchmarking on my old core2 processor using a single core pointed that 4 bytes would take 73 hours, too long, 3 bytes around 17 minutes, 2 bytes 4 seconds and 1 byte would be almost instantaneous.

I was aiming for something complex enough to be justifiable according to the history, but fast enough to be easy to playtest and solve. Originally I thought 3 byte keys could be great but still 17 minutes was too much time for a “random happy idea”, so I settled for using 2 byte ones but with a few more messages so that testing would require a little bit more of time.

The next step was transcribing the communications from the roof scene with a few additions, set up a plausible 10.0.0.0 network assignment for the different people communicating and put sctf{} around one of the lines so that it would be the flag.

A few lines of C code later I had a working PoC and after choosing the mobileip-agent port (for the extra meta) for the implementation I started Wireshark, created the pcap file and cracked it to verify my estimates and correctness.

The final step was making it clear that the participant was expected to crack a Blowfish cipher without making it too obvious (because otherwise the challenge would be really boring). Since in the movie the characters actually blow up the lobby of a government building and later go back to their ship, I decided to go for a short description which hopefully contained all the information needed to solve the challenge. In particular:

Whilst you were blowing up a government building and kicking agent asses I managed to intercept some of their internal communications. There is something fishy about the encryption they are using but I don’t know what.

My hope was that the use of the word fishy would raise some eyebrows and the reference to blowing the building would complete it. Sadly nobody contacted me to confirm if blowfish was indeed the algorithm so I decided to publish a hint with a distorted version of the image defining the algorithm. Unfortunately, by the end of the event nobody had cracked it. I think that, on retrospective, I should have added something mentioning clearly their use of short keys.

Intercepting Sentinels

The idea for this one came whilst I was working on the previous challenge. I was originally planning to use DES with a weak key but then the idea for weak blowfish came (and seemed to match better with the way you would communicate in an emulated world), so I was left with a non thematic challenge. Seeing my notes I noticed that on the third movie the sentinels used their antennas to receive commands from the machines and thus it came to me.

The machines would obviously need some way to securely communicate with each other, if possible something that could be implemented efficiently in hardware. Also 3DES came much later, in 1998 so taking into account that Matrix is based on the mid 90s (or so) it is reasonable to assume that humans will not have knowledge or resources to crack 56 bit keys.

Because of this I decided to use this algorithm for this challenge. The rest was simple, just write a small python utility that could encrypt the code using DES in ECB mode, and make a clear but not too obvious description. In this case I went for the following:

We know that the sentinels communicate using an standard to encrypt their data and managed to intercept one of their messages. We believe it is encrypted weakly. Can you help us decrypt it? Maybe we can confuse the sentinels before they breach the dock.

The reason for saying “encrypted weakly” is that DES has some keys (called possibly weak keys) which use only 4 subkeys (see page 21). It was one of these I chose for encrypting the flag.

Originally this was to be going documented in the tip we released for Agent Communications too, but as a team scored a solve I decided not to do so.

The Note

The problem with the two challenges above is that they are too hard for a newbie. It’s unlikely a newbie will try to bruteforce a 2 byte Blowfish cipher or know about the weak DES keys. So I needed to make a crypto-50 challenge.

In the first movie we see Cypher talking to Agent Smith in a restaurant, but the way in which the encounter was arranged is never explained. I decided to use a simple weakly encrypted note for this, something that the Operator may easily miss across all the cascading characters but that the Agents could easily decrypt.

Originally I was going to go for a Vigenère cipher, something simple enough for Cypher to prepare on the ship but hard enough for being easily noticed if smuggled into The Matrix. But whilst I was refreshing my knowledge of the cipher, I read about the Nihilist Cipher. This reminded me of the scene in which Neo gets a drive from a hollow book opened on the page talking “On Nihilism” which made the match a perfect one.

Once I picked keys related to Cypher (STEAK and BLUEPILL) and encoded the message with them, all I had left was the challenge description. Since a letter would be too obvious I decided to go with a book, so all I had left to do was finding one that would make your gears click and notice the cipher was the Nihilist one. Looking for books talking about Russian Nihilists I came up with the perfect match, Fathers and Sons by Ivan Turgenev (in russian Отцы и дети by Ива́н Серге́евич Турге́нев). And thus the challenge was born with the book referencing the cipher used.

The last flight of the Osiris

Whilst I was preparing the previous challenges I saw an interesting tweet in my stream. This made me think “It would be really cool to have a challenge using the events’ badge” and it came to me that a challenge where you had to decode the information of some “blinkenlights” could be a good idea. I tried to contact the conference organization in the hopes of loaning one to make this happen, sadly, we couldn’t get one in time so I fell back to using the one from the SHA2017 conference because its upython interface made it really easy to interface with it (specially since I had already coded some apps for it).

With six leds I had to find a way to display the data. The idea was making it slow enough for a human to be able to decode the message and with clear visual indicators. Thus I decided that having six leds, I could use one (the first) as the clocking indicator, one (the last) as a parity indicator to allow for error checking and the other 4 for displaying each specific nibble.

Originally, I was going to display the flag using ASCII but that would entail a zero on the first bit every two nibbles which could be confusing (as there was already a clock signal). Also I wanted something that screamed “obsolete technology” and then one of the Rooted Con challenges from my time at Little Nuns came to mind. In it they encoded the flag in the JUNK section of an avi file using EBCDIC to make it a bit less obvious. Therefore, I decided to go for EBCDIC for the encoding. I read about the different encodings and chose the Swedish one because the conference was in Sweden anyways (although sctf would encode the same in all of them).

With this I wrote the python code to display the leds, I recorded it with some trouble and uploaded to YouTube only to see YouTube decided to flip my video. After some reading I flipped the video back and looked for a song that could match the rhythm of the leds, the Matrix theme, and that was freely usable on YouTube. The aim, aside from making the video more effectful was mostly to hide the background noise of the recording. Finally, after a bit of editing and cutting the video was ready and published.

The last part was integrating this into the theme. I decided to make this be a beacon left on the Osiris that Niobe later found when scouting the remains because that would match well with the whole history.

Tracing Mr. Anderson

With the deadline approaching but having the crypto area covered I had some time left to focus on the “cool crazy ideas™”. At the end of the first Matrix movie, we see what looks like a call tracer trying to find the source of a call from Neo and failing. We can also see that phone tracing plays a very important role in the film too. Because of this the idea of making a challenge where participants had to trace calls from rebels seemed like a great idea.

The idea was making some kind of server where participants would get a pcap file and a question (for example which phone number is the call from 10.2.3.4 coming from) they had to answer in a short time (5 seconds or so). After enough correct answers they would get the flag. In the first levels the questions would be fixed but on the later ones they would be randomly chosen from a set. This with the aim of forcing the participants to make a tool able to actually trace any calls from the packet dumps.

I decided to use the SIP protocol because it was originally presented in 1996, is open, and is also easy enough to parse (as it is textual). I then wrote a SIP packet generator able to output to a pcap file using scapy. Whilst testing the tool I noticed that whilst 100 packets would be made reasonably fast, 1000 would take 4 seconds and the amount would then escalate linearly. This made it impossible to make the original challenge but I could instead make a forensics challenge where you were expected to find the odd packet instead.

The problem here was that I couldn’t just make the packet contain sctf{flag} because then a simple string search would bring this up. Instead, I decided to modify my code slightly to flaw the generator of User Agents (so that they would start with numbers from 1 to 9) and leave number 0 for the target packet (thus reading Phone0xxxx). The idea being that when visualizing the different fields the lack of a 0 would easily stick out in the graphs. After this I just used my generator to make a large pcap file so that participants wouldn’t just try to bruteforce the flag.

The last part was making it clear that they had to find the weird packet. I decided to make this by making the flag be the sha512 in hex format of the whole packet. Seeing that the wording could be a bit misleading I decided to clarify this further by publishing a tip.

The Oracle

This challenge was made during the conference after seeing that even with clues the number of solves for The Note was extremely low. In the film we see Trinity noting that she had visited The Oracle before but, we don’t see this in the films. Therefore, an encrypted file from the Oracle with her predictions for her seemed like an interesting way to deliver them (and allow me to make a challenge). Since the oracle is amazingly good at making predictions it would be reasonable to assume that she predicted that Trinity would eventually decrypt it.

I decided to use the xor monoalphabetic substitution cipher as it sounded like the kind of cipher a machine would use and was simple to crack. I quickly wrote down a note and a python script able to encrypt it and sent it for publication.

I wanted to be very clear about what was expected from the participant and I used the expression substitution cipher (on retrospective I should have used simple substitution or something similar to make it clear that it was not polyalphabetic), I also was a bit more open about tips and tools that could be useful because I wanted it to be as easily solvable as possible for newcomers.

On challenge difficulty

I would like to note that usually the hard part of writing CTF challenges is the level tuning. One lesson I learned from WannaTry (SecurityFest 2017 CTF) was that combining ideas makes difficulty grow exponentially, among other things because it makes it more likely for participants to go astray in the solve.

It is also very hard to write cryptographic challenges because your options basically wind up to either picking very specific and usually obscure cryptographical flaws, finding new ones, or going for simple things like classic ciphers. A way to make them a bit harder is by being a bit more obtuse in telling what you expect so that it is not crystal clear (and thus yet another boring crypto) but not excessively obscure either (and thus a guessing game). Tuning that is usually hard to do, specially when the level of your participants varies a lot as is the case here.

A good way to fix that is by having a few people proofing your challenges before the CTF but that is not always possible. In that case your best go is trying to put yourself in the mind of somebody trying to solve the challenge and hoping your clues are clear enough.

On challenge scoring

Scoring challenges is usually the hardest part. You have to define the challenge’s score based on the level you expect from the participants and what you have seen in other CTFs for similar challenges. In an ideal world you’d expect a 500 point challenge to be around 10 times more difficult than a 50 point one. The problem is that what you might perceive as easy (for example the Nihilist cypher) and may solve in 30 minutes, may take hours to others.

To try to address this issue, we have tested a slightly different approach. As you might have seen, likvidera wrote an amazing autoscoring system so that challenges would start at 500 points and slowly fall down to 50 as more solves were scored, this removed the scoring problem partially as we no longer need to manually score the challenges.

Still the problem of making a balanced CTF remains. You don’t want to have only 500 point challenges nor only 50 point ones, instead you want to have a variety of difficulties so that everybody can choose the level that best matches their experience. I usually try to do this by reminding the different stages of my learning and noting what I could and couldn’t do then. Also, over time you’ll also develop a certain intuition when estimating the difficulty of challenges which will allow you to see which is the difficulty they have and how composition will influence them.

Keep in mind though that independently of the scoring system you use, you should always have prepared a proof of concept of the solution. On one hand this will help you ensure that your challenges are correct, work as intended and can, therefore, be solved. On the other solving the challenge by yourself will help you estimate their difficulty.

Thanks

I would like to close this post thanking likvidera for organizing and arranging the CTF, the nice organization of Security Fest for their great conference and inviting us to attend and set their CTF and to b0bb, avlidienbrunn and d3vnu11 for making all the other challenges. Without them this post or the CTF would have never existed.

I would like to thank all those who have taken part and specially those who are writing amazing writeups of our challenges. You guys rock and make the effort worth it!

Finally, I would also like to thank you for reading this. Hopefully this will help you as a participant to become better at solving crypto and misc challenges and it will help you as an organizer to write better challenges on your CTFs.

Klondike from RJ45