Mr Gerald Giam Yean Song asked the Minister for Digital Development and Information with regard to the recent IT outage caused by CrowdStrike (a) whether the Cybersecurity Agency of Singapore (CSA) has updated its threat and risk assessment protocols to cover supply chain risks of this nature; (b) if so, whether these updated protocols will be implemented across all critical information infrastructure (CII); and (c) what new strategies are being considered to enhance the resilience of CIIs against systemic propagated shocks that are not directly linked to cybersecurity threats.
Mrs Josephine Teo: Mr Speaker, my response will also cover the matters raised in the Oral Questions by Assoc Prof Razwana Begum1,2,3, which are scheduled for a subsequent Sitting. With your permission, Sir, I would also like to invite all interested Members to seek clarifications after I have given my reply today. If the questions have been addressed, it may not be necessary to proceed with the Questions for future Sittings.
Mr Speaker: Please proceed.
Mrs Josephine Teo: Sir, on 19 July 2024, a faulty software update by cybersecurity service provider CrowdStrike disrupted major services around the world. Images of the now infamous “blue screen of death” appeared in media news cycles and attracted significant public attention. According to public reports, outages were experienced by users of the Microsoft Windows operating system that adopted CrowdStrike’s Falcon Endpoint Detection and Response (EDR) solution. It is a security solution that requires frequent and timely updates to be effective.
The Members’ questions fall broadly into two categories: first, what is the impact of the outage in Singapore, particularly in relation to services provided by the Government; second, what are the lessons learnt, particularly in relation to the resilience of our IT systems.
Fortuitously, Government services and most essential services in Singapore were unaffected by the outages. However, some businesses that use CrowdStrike’s Falcon EDR solution were affected. In most cases, the impact was to internal staff. In a minority of the cases, customers were impacted due to service disruptions. Prominent examples of these were the passenger check-in for some airlines at Changi Terminal 4 and gantry operations at some Housing and Development Board carparks.
Customers of affected businesses met with delays and were inconvenienced. However, business continuity plans kicked in. These included switching over to manual processes, such as for flight ticketing and check-in. The Singapore Cyber Emergency Response Team (SingCERT) of the Cyber Security Agency of Singapore (CSA) also quickly issued an advisory to guide affected systems administrators and users on how to manually recover their systems. Most of the affected systems recovered within a day and services returned to normal.
As Members know, IT systems may experience outages and disruptions from time to time. In this particular instance, it is not yet fully understood what caused a relatively routine software update to have created such major disruptions around the world. My Ministry has set up an internal task force to engage relevant partners to gain insights into the incident and assess if further measures should be taken to improve Singapore’s resilience when such disruptions occur.
In the meantime, one key lesson can already be reinforced. As we have said on previous occasions, even with best efforts, not all disruptions can be prevented. Systems owners should, therefore, have plans in place to help them recover quickly from unexpected disturbances.
On its part, the Government adopts a risk-based approach to ensure that our critical systems and Essential Services (ES) are resilient. Critical Information Infrastructures (CIIs), ES and Government services are all subject to stringent requirements and have to put in place robust business continuity plans, disaster recovery plans and incident response plans.
The Cybersecurity Act and specific sectoral regulations hold CII and key ES operators accountable for meeting the baseline security and resilience requirements. This includes timely review of risk assessments and audits. For example, Government agencies using third-party software in their information and communication technologies (ICT) systems have to do a thorough risk assessment and put in place the necessary mitigation measures. CSA also established the CII Supply Chain Programme to better manage key vendor supply chain risks.
Businesses must also play their part to improve their resilience when disruptions occur and recognise that it is in their own as well as their customers’ interests to do so. When things are running smoothly, businesses may question why they should incur cost or prioritise efforts to assess and improve their resilience measures. Unfortunately, some may not take appropriate action until it is too late.
We therefore encourage businesses to conduct their own risk management and assessment measures and put in place the appropriate business continuity plans to help business continuity in the event of a disruption. SingCERT has recently published an advisory on building digital resiliency, which can be found on CSA’s website. As part of the support for enterprises’ digitalisation, my Ministry offers other practical resources and financial assistance to encourage robust IT practices. This includes CSA’s cybersecurity toolkits and IMDA’s SMEs Go Digital Programme.
While these efforts may not specifically address IT outages like the one related to CrowdStrike, they can help businesses prevent incidents and recover more quickly should disruptions occur. I also encourage all businesses to take advantage of the Government’s resource support to strengthen their digital resilience.
Mr Speaker: Mr Alex Yam.
Mr Alex Yam (Marsiling-Yew Tee): Mr Speaker, I thank the hon Minister. The Minister mentioned that businesses should have contingencies in place. Could I ask the Minister, with regard to critical infrastructure, for example, businesses and airlines that are operating at our airports, whether we should have compulsory requirements? Because as we saw at our airport during the outage, most airlines were able to cope, whereas some airlines faced longer disruptions. This experience, although it is related to the airlines, does also reflect on Singapore on a whole. As such, will the Ministry consider making it compulsory for some businesses to adopt contingency plans?
Mrs Josephine Teo: Mr Speaker, actually, it is in the businesses’ own interest to have contingency plans in place. If they are affected, or if their customers are affected, certainly, the operational impact could be considerable. Certainly, their reputations are also at risk. The Government has to adopt a risk-based approach and that would include being quite careful about the occasions when we impose compulsory requirements.
If we attempt to prescribe the measures that businesses must take and we are not careful as to the occasions that we do so, firstly, it could take agency and the sense of ownership away from the IT systems’ owners, because then the thinking could be that, if the Government does not say so, then we do not need to do. That would be to the detriment of all of us.
Secondly, it is also, from a sense of humility, that we decide that this is not a good approach, because there are so many different components that go into a system’s resilience. To imagine that we have full understanding of all the different things that could cause major disruptions is, I believe, unwise.
I should also say that, in this particular instance, it was a fairly innocuous software update. No one could have expected the amount of disruption that it caused around the world. So, I would say that we will, in certain instances, require measures to be mandated. But in the vast majority of the cases, it is important to allow the systems owners and, indeed, to require the systems owners to take ownership, to build up their systems’ resilience. That is still the approach that we would prefer to take.
Mr Speaker: I see many hands up. I will call Members but let us keep the questions succinct and the answers likewise. Miss Cheryl Chan.
Miss Cheryl Chan Wei Ling (East Coast): Speaker, I have a supplementary question for the Minister. I agree with the Minister that these software systems are relatively complex and, generally, you do not just have a single supplier supplying to the overall system. The question would be, for the Government agencies, as we are going more digital or promoting digitalisation in our society, how can we ensure there is sufficient coordination with third-party suppliers, such that any change that they make on their independent end does not affect our overall system?
Mrs Josephine Teo: Sir, it is an excellent question. If I could seek your indulgence, it deserves a fuller response.
Firstly, the use of third-party software is unavoidable because technological systems are complex. Third-party software can offer a wide range of functionalities to meet the requirements of various organisations, the Government included, and this saves time and resources from having to develop such software from scratch. When using third-party software in their ICT systems, Government agencies are required to undergo a thorough risk assessment and to put in place the mitigating measures. That is already baseline.
To the extent possible, agencies must put in place quality assurance measures to ensure that the software changes that will be inevitable will not introduce errors in critical systems. Such measures include testing software updates in controlled settings prior to going live. IT people are very familiar with this – you test it in a controlled environment before you put it to the overall system and then see what happens.
We also deploy software changes progressively to small groups of users before rolling it out widely. This usually allows us to catch and isolate issues early. But I say “usually” because it does not happen all the time. There are ways in which the system components interact with each other that are not always possible to map out so clearly. In addition, agencies with critical systems are required to review the change management processes of their software providers through regular independent audits. This ensures that software changes can be rolled out smoothly and securely.
In some instances, depending on the service provided, it may be beyond the control of users, including governments. For example, Software as a Service will put the onus on the vendor to ensure that their software remains secure and available for use. This is something that we will have to keep in mind and see what we can do about.
To the Member’s specific questions – when we have exercises, for example, are different partners in the supply chain involved? Possibly, if they have a major impact on the system’s usability as well as resilience, but there are so many vendors involved and it may not be possible to include all of them in the exercises.
Mr Speaker: Mr Gerald Giam.
Mr Gerald Giam Yean Song (Aljunied): Sir, I have two supplementary questions for the Minister.
Sir, much of the legislation introduced recently covers cybersecurity risks. However, as this major outage caused by CrowdStrike was not a cybersecurity attack but a supply chain failure, specifically, a bug in the software update from the vendor, does the current legislation adequately address the risks posed by supply chain failures in digital infrastructure? And would the impending Digital Infrastructure Act have mitigated the impact of the outage caused by CrowdStrike?
Secondly, this incident also highlighted the risk of a single point of failure having widespread impact on digital infrastructure. A similar issue was observed with the Mobile Guardian mobile device management software used in schools which have affected thousands of students, many of whom discovered to their horror that they lost their study notes just before their weighted assessments. Is the Government looking at encouraging or even mandating operators of CII to review their IT procurement practices and diversify their sources of vendors so that no one software can bring down an entire system?
Mrs Josephine Teo: Mr Speaker, the hon Member has a number of questions rolled into his supplementary. Let me try and deal with supply chain risks more broadly and what we do about them in CII.
In fact, CSA’s threat and risk assessment for CII already cover supply chain risks of this nature today. So, it is not something that is new or unknown or we are caught by surprise in that regard. We have put in place measures to tackle the supply chain risks that the CIIs face, holistically.
For example, under the Cybersecurity Code of Practice, CII owners must adopt, to the extent possible, the Defence by Diversity principle. To explain to Members what this means – take any IT system. If the system does not have much diversity in terms of its defences and a single attack vector can immobilise the most key components of the system, then the system does not have Defence by Diversity. If the system wants to achieve Defence by Diversity, what the system owner needs to do is to have a different variety of vendors that the system owner works with, different system architectures to the extent possible, different configurations, different communications pathways and, indeed, different vendors for whether hardware or software.
So, Defence by Diversity is something that we ask CII owners to adopt to the extent possible. That is also why, in many instances, we emphasise how software systems ought to be interoperable. Because if interoperability is not common, then you can imagine that for system owners, they are stuck. Once system owners use system A, they must also use the other related systems for A. If they cannot choose to use B, then they do not have that diversity. So, that is a very important principle. It is already part of the way we operate.
The threat and risk assessments are also reviewed at regular intervals so that they remain up to date. Additionally, CSA has also introduced the CII Supply Chain Programme. I mentioned this earlier. This was in 2022. What does the Supply Chain Programme help, whether it is Government agencies or, indeed, any other system owner, achieve? It gives them a toolkit to help identify and inventorise their vendors.
It is very often the case that when something happens, system owners do not know what hit them, even if it has been reported in the media that this was a particular software failure, because the system owners may not know that their systems contain this particular software. So, a simple fact of inventorising what goes into your system is already no small feat because of the number of vendors involved along the whole supply chain.
So, the toolkit helps the systems owners to also assess and rate their cyber supply chain risk using a standardised vendor management methodology. So, that is another thing that we have put in place. To the Member’s point about a single point of failure, we agree and there are ways in which we mitigate against that.
For Government systems, critical functions are required to cater for redundancy. You must be able to have a failover, and this could include both the hardware and software components, networks and databases, as well as even aspects of the physical environment. So, this is something that is already practised and you can always improve them.
The Digital Infrastructure Act that we had talked about will seek to improve resilience that are over and above what needs to be done within the cyber system environment. We will have more details. We are in the process of consulting with the various stakeholders and, in due course, we will be able to say more about that.
Ministry of Digital Development and Information
7 August 2024
https://sprs.parl.gov.sg/search/#/sprs3topic?reportid=oral-answer-3614
Office of the Leader of the Opposition 