The ICS Detection Challenge is created by Dale Peterson, Eric Byres and aeSolutions to provide an objective test of the growing class of passive ICS detection solutions in the market. It will take place at S4x18, January 16-18 in Miami South Beach.
Phase I – Identification
The contestants will be provided with pcap files from span ports on a number of ICS switches and tasked with documenting the ICS to the greatest degree possible. These pcap files are from an actual ICS of moderate size and complexity in the oil/gas sector. The packets will be anonymized, but they are a representative, real world test.
Cyber asset identification will be at a variety of specificity levels with additional points granted for increased specificity. Bonus points can be won for correctly identifying cyber assets in a shorter time frame to recognize the difference between automated analysis and human analysis.
For example, basic points could be won by identifying a cyber asset as a PLC running a specific protocol. Additional points could be won by identifying the make and model of the PLC. Additional points could be won by identifying the firmware version. Additional points could be won by identifying what the PLC was monitoring and controlling.
Phase I will take place Tuesday morning (Jan 16th) and the results announced on the Main Stage on Tuesday afternoon.
The teams can continue analysis of the pcaps, if desired, to prepare for Phase II. This mirrors an environment where the detection team has increased time to learn the ICS and underlying process.
Phase II – Detection (Thursday)
A second set of PCAP files will be presented to contestants on Thursday morning. These PCAP files will include a variety of new communication that was not seen in the Phase I communication. The Phase II PCAPs will include a variety of attacks as well as new valid process communication that may be due to changes in the ICS or non-standard activities in the ICS.
Teams will win points by correctly identifying activity, again with additional points being earned based on specificity and timeliness of the results.
The results from Phase II will be announced in a Main Stage Session on Thursday afternoon. The winning teams from Phase I and Phase II will be invited on the Main Stage for an award ceremony and interview.
Points will be given for correct answers, and points will be deducted for incorrect answers.
The contestants will not be allowed to release the pcap files or use them for any purpose except for the S4 ICS Detection Challenge. Participants will be able to release their analysis of the pcaps, as well as Phase I and Phase II answers.
A preliminary information package will be provided on December 16th. This package will contain:
- A very high level description of the ICS that is the basis of the Challenge. Information will be restricted to basic information about the sector and process since Phase I is testing the identification capability.
- Information on how the PCAP files will be made available at the Challenge.
- Information on how answers must be presented, how the answers will be evaluated, and how answers will be scored.
- Additional information on logistics and rules.
There is no fee to participate in the ICS Detection Challenge, but all onsite participants must be registered to attend S4x18.
Registering To Compete
There are limited spots in this competition, and we will accept contestants as they are received until the spots are full. Contestants, after acceptance, have one week to register the team that will participate for S4x18. If the team is not registered in one week, the spot will be released.
Send your interest in competing in the S4 ICS Detection Challenge to firstname.lastname@example.org.