Strangers on a Train

“Mr. Trainman did (almost) all the things right, but …  something got terribly wrong”

This is a short story about how Industrial Control System Cybersecurity goes and the importance of real ICS Cybersecurity companies and professionals. A use case on the railway sector.

Landscape and main characters

Once upon a time, a National Railway company was building a new connection line between two main cities. Public tender for train provisioning on that new line containing Safety and Cybersecurity requirements for new equipment was published.

There was a train vendor manufacturing last generation trains all around the world. with leading edge technology on board equipment to add differential value, always complaining with different Safety and Cybersecurity regulations. Mr. Trainman as corporate CISO in front of these issues.

Nowadays, different networks and systems can be found on board in a train. one of those is CCTV (Inside and Outside):

For this project, they were looking for a unique vendor to provide both solutions (Inside and Outside) on CCTV system, to simplify project management and reduce complexity.

Then Moxa came in as they could provide both solutions, especially the outside CCTV needed for safety on persons going up and down on stops.

Having in mind all the Cybersecurity requirements, Mr trainman had to review that vendor selection. 

Mr. Trainman validation checklist

Being a regulated sector, all onboard equipment has to be complain with some railway safety standards:

But as a cybersecurity professional, Mr. Trainman also had some vendor requirements on the Cybersecurity posture, and he was looking for a true secure product development lifecycle certified organization:

With this on mind, Mr, Trainman began to check all these requirements on the selected vendor and product:

Good !!!!

Safety certifications and Ethernet protocols documented to place the equipment on the right IEC 62433 zone inside train network. Good !!!!

Some security controls available to protect the device:

• Password access protection
• IP Filtering
• HTTPS (Encryption) supported

Good !!!!!

Vendor and equipment approved and inside the new train to be delivered to customer.

Mr. Trainman met his stranger on a train

On May 2021, everything changed:

Four critical vulnerabilities on Outside Moxa IP Camera were published by NIST, and:

Vulnerabilities on product leads to a Denial of Service 

(CRITICAL for customer and safety) 

Firmware update MANDATORY on any already deployed camera

Vulnerabilities come from a Level 2 protocol (LLDP)

LLDP not identified by vendor on Spec Sheets

Level 2 filtering was not on Zoning Firewall rules

A complete revision on train IEC 62433 security Zoning needed ASAP

Now they were facing a huge economic impact on the project due to execution of all the tasks describes above.

Some final advices for Mr. Trainman

• Certification is good, but Not Enough
• Cybersecurity Certification Framework still to come on some sectors. Safety regulations are not security regulations. 

• Don’t trust, TEST because:
• Vendor spec sheets, and even technical manuals, don’t describe all the network ports and protocols products use.
• 0-Day vulnerabilities exists even if none had already tested that product, so do it yourself.
• Testing before deploying is cost saving and investment protection
• Always test your new products with an experienced Industrial Control System Cybersecurity Company

Dragonfly 2.0: Testing the Watering Hole risk

Three years after Dragonfly 1.0, now we are facing Dragonfly 2.0. In both campaigns, Watering Hole is one of the attack vectors described in reports, so protecting the website accessed by ICS engineers to get important resources from their daily work should be very important. It seems like ICS vendors and asset owners should have learned from that, and since I am very keen of testing things, I have decide to check that.
I designed a simple test to check the security posture from 13 big ICS vendors on communication security protection to their support web sites. (There is where the firmware upgrades used to be nowadays).

Mayor ICS Vendors I tested were:

• Siemens
• Schneider Electric
• Honeywell
• Rockwell Automation
• Yokogawa
• Moxa
• OSIsoft
• Phoenix Contact
• Advantech
• SEL
• ABB
• CODESYS
• MatrikonOPC

To make this checks, I used a simple but powerful tool called testssl.sh (that BTW, I recommend to you). This tool is able of testing, between other things, the following SSL vulnerabilities:

• Heartbleed (CVE-2014-0160)
• CCS (CVE-2014-0224)
• Secure Renegotiation (CVE-2009-3555)
• Secure Client-Initiated Renegotiation
• CRIME, TLS (CVE-2012-4929)
• BREACH (CVE-2013-3587)
• POODLE, SSL (CVE-2014-3566) 
• TLS_FALLBACK_SCSV (RFC 7507),
• FREAK (CVE-2015-0204)
• DROWN (2016-0800, CVE-2016-0703)
• LOGJAM (CVE-2015-4000)
• BEAST (CVE-2011-3389) 
• RC4 (CVE-2013-2566, CVE-2015-2808)

When using the tool against and SSL protected website, you can get "Vulnerable", "Probably" or "Not Vulnerable" results, that I associate with 2,1 and 0 values. That way, the most vulnerabilities the tool found for each SSL support web page, the higher risk I associate with the vendor.

These are the results for vulnerabilities:

First finding is that Advantech dosen't even have an SSL protected support Web page, So I associate the maximum risk value to it.

Second finding is that big ICS vendors were very similar in vulnerabilities, apart from ABB that scores much better with only one probably fixed vulnerability. Honeywell, on the other hand, shows the most potential problems to fix.

With these values on mind, I established another criteria to build a Heat Map (I love Heat Maps).
Seems logical thinking that this kind of risk is directly proportional with the use of the support site, so I searched for the Alexa Rank of those ICS support pages, and these are the values I found that day:

The lowest rank, the most accesses ...

Normalizing values from 1 to 5, I got the following Heat Map :

Seems like someone has some homework to do ....

Keep tune till the next revision.

Default passwords on control systems: SCADAPASS + DPE = Raise Awareness

When talking about responsible disclosure, everybody has an strong opinion. You can think publishing vulnerabilities is an irresponsible act, or you can think it is the only way of raise awareness on assets owners.
I am in the second category. As a control system Cybersecurity evangelist, I spend many time trying to show the hidden attack surface many organizations have, and the lack of visibility on the risks they are facing by managing their control systems totally apart from the Cybersecurity management framework other systems are in.
SAP servers, File servers, switches and firewalls on the corporate network have security policies, regular audits and security log and monitoring. Not doing that should be considered "irresponsible" and negligent.
The same corporation is using CCTV surveillance, access control systems, HVAC, fire detection system, lights and energy management systems and many other critical support systems, deployed by poorly security practices trained control integrators.  And now, both networks are connected. (And some times, Hyper-connected)

One of the worst (and common) practices you can find in control networks is installing by default. It means default usernames and passwords works in many devices inside the control network.

On December 2015, SCADA StrangeLove put in place a Default password publishing initiative, called SCADAPASS to rise awareness on control assets owners. Apart from your feelings or opinion on this, impact was huge in the ICS Cybersecurity world and it was included on Metasploit 4.11.6 .

Recently I remembered DPE project from ToolsWatch , and after getting familiar with dpeparser.py, I decided to integrate SCADA StrangeLove results on the XML db.
I wrote a quick-piggy script to convert CSV to DPE XML Schema and these are my results. All new entries are in the "sacda" type (-t scada).

As we love numbers, this is a summary from my work:

• About 47 new pure ICS vendors on DB (Siemens and Advantech were already there):
• 125 new elements (Without CPE, nor CVE since this is a harder process as you may know), but with their control role or description on the network. The list is too long to publish here.
• 203 new default usernames and passwords. Some devices has more than 50 default usernames !!!

I have contacted with the DPE development team, as I think it can be very useful to anyone in charge of assessing or managing control networks, but most important for assets owners of such networks.

New ICS DPE file can be downloaded here

Low cost SCADA Assessment: First internal approach

Best things in life are for free. That's what they say and sometimes I agree.

I read a post from EduHacker talking about Google fusion tables and it network graph functionality, so I decided give it a try.

I always try to quickly draw the traffic matrix from the network we are assessing since bandwidth between eyes and brain is the biggest you can get, and frankly most of the times I need it.

As ICS engineers we always use passive methods to assess through mirroring network traffic on switches. (By the way, this is a risky business since you have to be completely sure switch is not going to restart by configuring that).

Once you have your mirror port you can then plug your laptop and begin gathering IP traffic. It is amazing what you can get in a few minutes.

I took one ten minutes pcap file from one of our last OASyS SCADA assessments and follow the instructions from the post I read.

First of all you have to convert your pcap file to comma separated values (CSV) to import it in a Google fusion table.

To do this I made a quick and dirty csh script (I know, but I am a classic sir):

#!/bin/csh -f

###########################################################

#

# (c) 2015 - Kaostopper

#

# pcap_to_csv.csh: Quick and dirty way of getting a CSV file

#                  from a pcap file

#

#

# Enrique Martin Garcia

#

###########################################################

if ( $#argv != 1 ) then

  echo "Usaage: $0 pcap_file"

  exit 1

endif

set fichero_pcap = "$1"

set TSHARK = /usr/local/bin/tshark

echo "src_ip,src_port,dst_ip,dst_port,ip_proto"

$TSHARK -n -r $fichero_pcap -T fields -E separator=, -e ip.src -e tcp.srcport -e ip.dst -e tcp.dstport -e _ws.col.Protocol | grep -v ",," | sort -u

$TSHARK -n -r $fichero_pcap -T fields -E separator=, -e ip.src -e udp.srcport -e ip.dst -e udp.dstport -e _ws.col.Protocol | grep -v ",," | sort -u

exit 0

You can use any other way to convert your pcap files and you can choose any other fields, of course.

You have to be careful after uploading your CSV file to a fusion table since it is necessary to fix the type of value for your src_ip and dst_ip columns to text.

To do so, you have to click on the column header to access the change option:

You have to fix the type to text on those columns (src_ip and dst_ip)

With those two changes done, you can create your network graph:

As you can see, it was easy to detect a big misconfiguration in the Operation Workstation that was trying to connect to Google DNS servers. (8.8.8.8 and 8.8.4.4). This came from a temporary movement of this station due to civil works on the control room, but .....

You can play a lot with the other different types of graph, but this one is very easy to get and you don't need any other tool.

Beware with the data you upload and do like me: I have changed everything on the CSV file except public Google addresses and the SCADA System name.

No more excuses to update a very simple but useful network diagram for your SCADA Network.

Cyber physical attacks to critical infrastructure (Part I: Critical Infrastructure Systems domain)

Is it just another commercial trick from ICS Cyber Security sector or something to take care about ?

From the Aurora experiment cyber attack on a power generator in 2007, which was intended to demonstrate the ability to produce physical damage to assets remotely, to this day, this type of attack has materialized twice. (As far as we know).

The first cyber-physical attack in history detected, documented and widely known in the field of industrial cyber security professionals was STUXNET (2010), which marked the beginning of the development of this discipline and most standards for critical infrastructure, as it demonstrated the enormous destructive power of malware aimed at the destruction of the uranium enrichment centrifuges that Iran would use in its production of nuclear weapons.

The second cyber attack with physical consequences occurred recently (end of 2014) in a German steel plant, in which a cyber attack triggered after accessing control network from the business network, did not allow a graceful shutdown for a blast furnace, although the details and effects thereof have not been studied with the same detail as in the case of STUXNET.

In 2015 the interest on such attacks focus in altering the physical behavior of the environment through cyber attacks has increased through experiments carried out on cars, medical instruments and numerous automation devices connected to the Internet.

In the last Black Hat 2015 USA and DEF CON 23, there were very interesting presentations about these Cyber Physical Attacks from Marina Krotofil and Jason Larsen describing many of them.

This post, and the next ones, studies the higher impact (and therefore riskier) attacks on cyber-physical systems in critical infrastructure control networks and propose protection by making some changes on organizations structures and procedures and new technologies of intrusion detection based on analysis behavior of control protocols and correlation of operational events.

Critical sector and critical infrastructure

To put into context the domain to protect from such attacks, we describe the characteristics considered critical infrastructure in Europe and in Spain.

In January 2009 it came into effect Directive 2008/114 / EC of the Council of the European Union which established the need to identify Europe's critical infrastructures in order to design strategies to protect them.

In this Directive the need to identify infrastructure sectors of energy and transport, leaving open the possibility that all member states identify additional critical sectors.

As of December 2014 the European Agency for Network Security and Information Agency (ENISA) published a guide for the identification of critical assets.

This guide showed critical sectors  already identified by the member countries of the Union and can be seen in the following table:

Spain has identified twelve critical sectors:

• Energy (With three subsectors: Electricity, Oil and Gas)
• Nuclear
• Economics (Finance and Tax Administration)
• Water
• Transportation (With three subsectors: air, sea and land)
• Food
• Information Technologies and Communications
• Chemical
• Health
• Space
• Public administration
• Research

In each of these sectors they have been appointed or will be appointed in the near future, a set of Critical Operators (OC), which are those owners or operators of infrastructures which provide essential services and whose attack could lead to damages broad sectors of the population. This set of infrastructures will shape our domain to protect and share a number of common technical characteristics.

Technical characteristics. 

Many classified as critical infrastructure have a hybrid architecture in which there are networks of classical information technology (IT Network) and industrial control networks (OT Network) managing the elements that interact with the physical environment (cyber-physical systems). A basic scheme of this type of infrastructure could be the following:

  

Cyber-physical systems control a particular process and are managed by network systems, operate according to the following basic scheme:

The sensors measure the current process values on fixed intervals and send them to the control units assessing the need to seek concrete solutions to the actuators orders that the process remains within the values for which it was created and behave according to the original design.

Today all this traffic control has been migrating to TCP networks and conventional operating systems, which has made no earlier existing attack surfaces appear.

The key characteristics of the OT networks can be summarized as follows:

• Less number of devices and services than IT networks. 
• They should never be directly connected to Internet. 
• Execute repetitive operations between its nodes and systems.
• Very sensitive to delays or communication problems.

But these classes of networks also have strong weakness as:

• Use insecure or unauthenticated protocols. 
• Often not segmented logically or physically. 
• No possibility of installing third party software on some systems. 
• No possibility of patching or update certain systems

These features and constraints make protection of such critical networks very special and, as discussed below, using specific strategies and technologies for this type of environment.

In the next post we will dive on that.