Apr 16, 2024NewsroomCloud Security / DevSecOps

New cybersecurity research has found that command-line interface (CLI) tools from Amazon Web Services (AWS) and Google Cloud can expose sensitive credentials in build logs, posing significant risks to organizations.

The vulnerability has been codenamed LeakyCLI by cloud security firm Orca.

“Some commands on Azure CLI, AWS CLI, and Google Cloud CLI can expose sensitive information in the form of environment variables, which can be collected by adversaries when published by tools such as GitHub Actions,” security researcher Roi Nisimi said in a report shared with The Hacker News.

Microsoft has since addressed the issue as part of security updates released in November 2023, assigned it the CVE identifier CVE-2023-36052 (CVSS score: 8.6).

The idea, in a nutshell, has to do with how the CLI commands such as could be used to show (pre-)defined environment variables and output to Continuous Integration and Continuous Deployment (CI/CD) logs. A list of such commands spanning AWS and Google Cloud is below 0

• aws lambda get-function-configuration
• aws lambda get-function
• aws lambda update-function-configuration
• aws lambda update-function-code
• aws lambda publish-version
• gcloud functions deploy <func> –set-env-vars
• gcloud functions deploy <func> –update-env-vars
• gcloud functions deploy <func> –remove-env-vars

Orca said it found several projects on GitHub that inadvertently leaked access tokens and other sensitive data via Github Actions, CircleCI, TravisCI, and Cloud Build logs.

Unlike Microsoft, however, both Amazon and Google consider this to be expected behavior, requiring that organizations take steps to avoid storing secrets in environment variables and instead use a dedicated secrets store service like AWS Secrets Manager or Google Cloud Secret Manager.

Google also recommends the use of the “–no-user-output-enabled” option to suppress the printing of command output to standard output and standard error in the terminal.

“If bad actors get their hands on these environment variables, this could potentially lead to view sensitive information including credentials, such as passwords, user names, and keys, which could allow them to access any resources that the repository owners can,” Nisimi said.

“CLI commands are by default assumed to be running in a secure environment, but coupled with CI/CD pipelines, they may pose a security threat.”

Apr 16, 2024NewsroomThreat Intelligence / Endpoint Security

The threat actor tracked as TA558 has been observed leveraging steganography as an obfuscation technique to deliver a wide range of malware such as Agent Tesla, FormBook, Remcos RAT, LokiBot, GuLoader, Snake Keylogger, and XWorm, among others.

“The group made extensive use of steganography by sending VBSs, PowerShell code, as well as RTF documents with an embedded exploit, inside images and text files,” Russian cybersecurity company Positive Technologies said in a Monday report.

The campaign has been codenamed SteganoAmor for its reliance on steganography and the choice of file names such as greatloverstory.vbs and easytolove.vbs.

A majority of the attacks have targeted industrial, services, public, electric power, and construction sectors in Latin American countries, although companies located in Russia, Romania, and Turkey have also been singled out.

The development comes as TA558 has also been spotted deploying Venom RAT via phishing attacks aimed at enterprises located in Spain, Mexico, the United States, Colombia, Portugal, Brazil, Dominican Republic, and Argentina.

It all starts with a phishing email containing a booby-trapped email Microsoft Excel attachment that exploits a now-patched security flaw in Equation Editor (CVE-2017-11882) to download a Visual Basic Script that, in turn, fetches the next-stage payload from paste[.]ee.

The obfuscated malicious code takes care of downloading two images from an external URL that come embedded with a Base64-encoded component that ultimately retrieves and executes the Agent Tesla malware on the compromised host.

Beyond Agent Tesla, other variants of the attack chain have led to an assortment of malware such as FormBook, GuLoader, LokiBot, Remcos RAT, Snake Keylogger, and XWorm, which are designed for remote access, data theft, and delivery of secondary payloads.

The phishing emails are sent from legitimate-but-compromised SMTP servers to lend the messages a little credibility and minimize the chances of them getting blocked by email gateways. In addition, TA558 has been found to use infected FTP servers to stage the stolen data.

The disclosure comes against the backdrop of a series of phishing attacks targeting government organizations in Russia, Belarus, Kazakhstan, Uzbekistan, Kyrgyzstan, Tajikistan, and Armenia with a malware dubbed LazyStealer to harvest credentials from Google Chrome.

Positive Technologies is tracking the activity cluster under the name Lazy Koala in reference to the name of the user (joekoala), who is said to control the Telegram bots that receive the stolen data.

That said, the victim geography and the malware artifacts indicate potential links to another hacking group tracked by Cisco Talos under the name YoroTrooper (aka SturgeonPhisher).

“The group’s main tool is a primitive stealer, whose protection helps to evade detection, slow down analysis, grab all the stolen data, and send it to Telegram, which has been gaining popularity with malicious actors by the year,” security researcher Vladislav Lunin said.

The findings also follow a wave of social engineering campaigns that are designed to propagate malware families like FatalRAT and SolarMarker.

Apr 16, 2024NewsroomSupply Chain / Software Security

Security researchers have uncovered a “credible” takeover attempt targeting the OpenJS Foundation in a manner that evokes similarities to the recently uncovered incident aimed at the open-source XZ Utils project.

“The OpenJS Foundation Cross Project Council received a suspicious series of emails with similar messages, bearing different names and overlapping GitHub-associated emails,” OpenJS Foundation and Open Source Security Foundation (OpenSSF) said in a joint alert.

According to Robin Bender Ginn, executive director of OpenJS Foundation, and Omkhar Arasaratnam, general manager at OpenSSF, the email messages urged OpenJS to take action to update one of its popular JavaScript projects to remediate critical vulnerabilities without providing any specifics.

The email author(s) also called on OpenJS to designate them as a new maintainer of the project despite having little prior involvement. Two other popular JavaScript projects not hosted by OpenJS are also said to have been at the receiving end of similar activity.

That said, none of the people who contacted OpenJS were granted privileged access to the OpenJS-hosted project.

The incident brings into sharp focus the method by which the lone maintainer of XZ Utils was targeted by fictitious personas that were expressly created for what’s believed to be a social engineering-cum-pressure campaign designed to make Jia Tan (aka JiaT75) a co-maintainer of the project.

This has raised the possibility that the attempt to sabotage XZ Utils may not be an isolated incident and that it’s part of a broader campaign to undermine the security of various projects, the two open source groups said. The names of the JavaScript projects were not disclosed.

Jia Tan, as it stands, has no other digital footprints outside of their contributions, indicating that the account was invented for the sole purpose of gaining the credibility of the open-source development community over years and ultimately push a stealthy backdoor into XZ Utils.

It also serves to pinpoint the sophistication and patience that has gone behind planning and executing the campaign by targeting an open-source, volunteer-run project that’s used in many Linux distributions, putting organizations and users at risk of supply chain attacks.

The XZ Utils backdoor incident also highlights the “fragility” of the open-source ecosystem and the risks created by maintainer burnout, the U.S. Cybersecurity and Infrastructure Security Agency (CISA) said last week.

“The burden of security shouldn’t fall on an individual open-source maintainer — as it did in this case to near-disastrous effect,” CISA officials Jack Cable and Aeva Black said.

“Every technology manufacturer that profits from open source software must do their part by being responsible consumers of and sustainable contributors to the open source packages they depend on.”

The agency is recommending that technology manufacturers and system operators that incorporate open-source components should either directly or support the maintainers in periodically auditing the source code, eliminating entire classes of vulnerabilities, and implementing other secure by design principles.

“These social engineering attacks are exploiting the sense of duty that maintainers have with their project and community in order to manipulate them,” Bender Ginn and Arasaratnam said.

“Pay attention to how interactions make you feel. Interactions that create self-doubt, feelings of inadequacy, of not doing enough for the project, etc. might be part of a social engineering attack.”

Apr 16, 2024The Hacker NewsCloud Security / Threat Intelligence

In today’s rapidly evolving digital landscape, organizations face an increasingly complex array of cybersecurity threats. The proliferation of cloud services and remote work arrangements has heightened the vulnerability of digital identities to exploitation, making it imperative for businesses to fortify their identity security measures.

Our recent research report, The Identity Underground Report, offers valuable insights into the challenges and vulnerabilities organizations encounter in managing digital identities. The report paints a vivid picture of the “hidden” identity security liabilities where attackers leverage Identity Threat Exposures (ITEs) such as forgotten user accounts and misconfigurations to breach organizations’ defenses, with each ITE posing a significant threat to organizations’ security posture.

These findings reveal alarming statistics that underscore the widespread prevalence of ITEs across organizations of all sizes:

• 67% of organizations unknowingly expose their SaaS applications to potential compromise through insecure password synchronization practices.
• 37% of admin users still rely on weak authentication protocols like NTLM.
• 31% of user accounts are service accounts, which attackers seek to target as security teams often overlook them.
• A single misconfiguration in Active Directory spawns an average of 109 new shadow admins, enabling attackers to change settings and permissions, and gain more access to machines as they move deeper into an environment.

The shift to cloud-based environments introduces additional challenges, as organizations synchronize on-prem user accounts with cloud Identity Providers (IdPs). While this streamlines access, it also creates a pathway for attackers to exploit ITEs in on-prem settings to gain unauthorized access to cloud resources.

Ultimately, it is essential to recognize the dynamic nature of identity threats. Cybercriminals are constantly evolving their tactics, underscoring the need for a holistic and layered approach to security. By adopting proactive measures like Multi-Factor Authentication (MFA) and investing in robust identity security solutions, organizations can enhance their resilience against identity-related threats.

Learn more about the underground weaknesses that expose organizations to identity threats here and heed the report’s findings to prioritize security investments and eliminate your identity security blind spots.

Apr 16, 2024NewsroomEncryption / Network Security

The maintainers of the PuTTY Secure Shell (SSH) and Telnet client are alerting users of a critical vulnerability impacting versions from 0.68 through 0.80 that could be exploited to achieve full recovery of NIST P-521 (ecdsa-sha2-nistp521) private keys.

The flaw has been assigned the CVE identifier CVE-2024-31497, with the discovery credited to researchers Fabian Bäumer and Marcus Brinkmann of the Ruhr University Bochum.

“The effect of the vulnerability is to compromise the private key,” the PuTTY project said in an advisory.

“An attacker in possession of a few dozen signed messages and the public key has enough information to recover the private key, and then forge signatures as if they were from you, allowing them to (for instance) log in to any servers you use that key for.”

However, in order to obtain the signatures, an attacker will have to compromise the server for which the key is used to authenticate to.

In a message posted on the Open Source Software Security (oss-sec) mailing list, Bäumer described the flaw as stemming from the generation of biased ECDSA cryptographic nonces, which could enable the recovery of the private key.

“The first 9 bits of each ECDSA nonce are zero,” Bäumer explained. “This allows for full secret key recovery in roughly 60 signatures by using state-of-the-art techniques.”

“These signatures can either be harvested by a malicious server (man-in-the-middle attacks are not possible given that clients do not transmit their signature in the clear) or from any other source, e.g. signed git commits through forwarded agents.”

Besides impacting PuTTY, it also affects other products that incorporate a vulnerable version of the software –

• FileZilla (3.24.1 – 3.66.5)
• WinSCP (5.9.5 – 6.3.2)
• TortoiseGit (2.4.0.2 – 2.15.0)
• TortoiseSVN (1.10.0 – 1.14.6)

Following responsible disclosure, the issue has been addressed in PuTTY 0.81, FileZilla 3.67.0, WinSCP 6.3.3, and TortoiseGit 2.15.0.1. Users of TortoiseSVN are recommended to use Plink from the latest PuTTY 0.81 release when accessing an SVN repository via SSH until a patch becomes available.

Specifically, it has been resolved by switching to the RFC 6979 technique for all DSA and ECDSA key types, abandoning its earlier method of deriving the nonce using a deterministic approach that, while avoiding the need for a source of high-quality randomness, was susceptible to biased nonces when using P-521.

On top of that, ECDSA NIST-P521 keys used with any of the vulnerable components should be considered compromised and consequently revoked by removing them from authorized_keys files files and their equivalents in other SSH servers.

Apr 16, 2024NewsroomPrivacy Breach / Regulatory Compliance

The U.S. Federal Trade Commission (FTC) has ordered the mental telehealth company Cerebral from using or disclosing personal data for advertising purposes.

It has also been fined more than $7 million over charges that it revealed users’ sensitive personal health information and other data to third parties for advertising purposes and failed to honor its easy cancellation policies.

“Cerebral and its former CEO, Kyle Robertson, repeatedly broke their privacy promises to consumers and misled them about the company’s cancellation policies,” the FTC said in a press statement.

While claiming to offer “safe, secure, and discreet” services in order to get consumers to sign up and provide their data, the company, FTC alleged, did not clearly disclose that the information would be shared with third-parties for advertising.

The agency also accused the company of burying its data sharing practices in dense privacy policies, with the company engaging in deceptive practices by claiming that it would not share users’ data without their consent.

The company is said to have provided the sensitive information of nearly 3.2 million consumers to third parties such as LinkedIn, Snapchat, and TikTok by integrating tracking tools within its websites and apps that are designed to provide advertising and data analytics functions.

The information included names; medical and prescription histories; home and email addresses; phone numbers; birthdates; demographic information; IP addresses; pharmacy and health insurance information; and other health information.

The FTC complaint further accused Cerebral of failing to enforce adequate security guardrails by allowing former employees to access users’ medical records from May to December 2021, using insecure access methods that exposed patient information, and not restricting access to consumer data to only those employees who needed it.

“Cerebral sent out promotional postcards, which were not in envelopes, to over 6,000 patients that included their names and language that appeared to reveal their diagnosis and treatment to anyone who saw the postcards,” the FTC said.

Pursuant to the proposed order, which is pending approval from a federal court, the company has been barred from using or disclosing consumers’ personal and health information to third-parties for marketing, and has been ordered to implement a comprehensive privacy and data security program.

Cerebral has also been asked to post a notice on its website alerting users of the FTC order, as well as adopt a data retention schedule and delete most consumer data not used for treatment, payment, or health care operations unless they have consented to it. It’s also required to provide a mechanism for users to get their data deleted.

The development comes days after alcohol addiction treatment firm Monument was prohibited by the FTC from disclosing health information to third-party platforms such as Google and Meta for advertising without users’ permission between 2020 and 2022 despite claiming such data would be “100% confidential.”

The New York-based company has been ordered to notify users about the disclosure of their health information to third parties and ensure that all the shared data has been deleted.

“Monument failed to ensure it was complying with its promises and in fact disclosed users’ health information to third-party advertising platforms, including highly sensitive data that revealed that its customers were receiving help to recover from their addiction to alcohol,” FTC said.

Over the past year, FTC has announced similar enforcement actions against healthcare service providers like BetterHelp, GoodRx, and Premom for sharing users’ data with third-party analytics and social media firms without their consent.

It also warned [PDF] Amazon against using patient data for marketing purposes after it finalized a $3.9 billion acquisition of membership-based primary care practice One Medical.

Two individuals have been arrested in Australia and the U.S. in connection with an alleged scheme to develop and distribute a remote access trojan called Hive RAT (previously Firebird).

The U.S. Justice Department (DoJ) said the malware “gave the malware purchasers control over victim computers and enabled them to access victims’ private communications, their login credentials, and other personal information.”

A 24-year-old individual named Edmond Chakhmakhchyan (aka “Corruption”) from Van Nuys in Los Angeles, California, was taken into custody after he was caught selling a license of Hive RAT to an undercover employee of a law enforcement agency.

He has been charged with one count of conspiracy and one count of advertising a device as an interception device, each of which carries a penalty of five years in prison. Chakhmakhchyan pleaded not guilty and was ordered to stand trial on June 4, 2024.

Court documents allege a partnership between the malware’s creator and the defendant under which the latter would post advertisements for the malware on a cybercrime forum called Hack Forums, accept cryptocurrency payments from customers, and offer product support.

Hive RAT comes with capabilities to terminate programs, browse files, record keystrokes, access incoming and outgoing communications, and steal victim passwords and other credentials for bank accounts and cryptocurrency wallets from victims’ machines without their knowledge or consent.

“Chakhmakhchyan exchanged electronic messages with purchasers and explained to one buyer that the malware ‘allowed the Hive RAT user to access another person’s computer without that person knowing about the access,'” the DoJ said.

The Australian Federal Police (AFP), which announced charges of its own against a citizen for their purported involvement in the creation and sale of Hive RAT, said its investigation into the matter began in 2020.

The unnamed suspect faces 12 charges, including one count of producing data with intent to commit a computer offense, one count of controlling data with intent to commit a computer offense, and 10 counts of supplying data with intent to commit a computer offense. The maximum penalty for each of these offenses is three years imprisonment.

“Remote Access Trojans are one of the most harmful cyber threats in the online environment – once installed onto a device, a RAT can provide criminals with full access to, and control of the device,” AFP Acting Commander Cybercrime Sue Evans said.

“This could include anything from committing crimes anonymously, watching victims through camera devices, wiping hard drives, or stealing banking credentials and other sensitive information.”

Nebraska Man Indicted in Cryptojacking Scheme

The development comes as federal prosecutors in the U.S. indicted Charles O. Parks III (aka “CP3O”), 45, for operating a massive illegal cryptojacking operation, defrauding “two well-known providers of cloud computing services” out of more than $3.5 million in computing resources to mine cryptocurrency worth nearly $1 million.

The indictment charges the Parks with wire fraud, money laundering, and engaging in unlawful monetary transactions. He was arrested on April 13, 2024. The wire fraud and money laundering charges carry a maximum sentence of 20 years’ imprisonment. He also faces a 10 years’ imprisonment on the unlawful monetary transactions charges.

While the DoJ does not explicitly state what cloud providers were targeted in the fraudulent operation, it noted that the companies are based in the Washington state cities of Seattle and Redmond – the corporate headquarters for Amazon and Microsoft.

“From in or about January 2021 through August 2021, Parks created and used a variety of names, corporate affiliations and email addresses, including emails with domains from corporate entities he operated […] to register numerous accounts with the cloud providers and to gain access to massive amounts of computing processing power and storage that he did not pay for,” the DoJ said.

The illicitly obtained resources were then used to mine cryptocurrencies such as Ether (ETH), Litecoin (LTC) and Monero (XMR), which were laundered through a network of cryptocurrency exchanges, a non-fungible token (NFT) marketplace, an online payment provider, and traditional bank accounts to conceal digital transaction trail.

The ill-gotten proceeds, prosecutors said, were ultimately converted into dollars, which Parks used to make various extravagant purchases that included a Mercedes Benz luxury car, jewelry, and first-class hotel and travel expenses.

“Parks tricked the providers into approving heightened privileges and benefits, including elevated levels of cloud computing services and deferred billing accommodations, and deflected inquiries from the providers regarding questionable data usage and mounting unpaid subscription balances,” the DoJ said.

Apr 15, 2024NewsroomFirmware Security / Vulnerability

A security flaw impacting the Lighttpd web server used in baseboard management controllers (BMCs) has remained unpatched by device vendors like Intel and Lenovo, new findings from Binarly reveal.

While the original shortcoming was discovered and patched by the Lighttpd maintainers way back in August 2018 with version 1.4.51, the lack of a CVE identifier or an advisory meant that it was overlooked by developers of AMI MegaRAC BMC, ultimately ending up in products made by Intel and Lenovo.

Lighttpd (pronounced “Lighty”) is an open-source high-performance web server software designed for speed, security, and flexibility, while optimized for high-performance environments without consuming a lot of system resources.

The silent fix for Lighttpd concerns an out-of-bounds read vulnerability that could be exploited to exfiltrate sensitive data, such as process memory addresses, thereby allowing threat actors to bypass crucial security mechanisms like address space layout randomization (ASLR).

“The absence of prompt and important information about security fixes prevents proper handling of these fixes down both the firmware and software supply chains,” the firmware security company said.

The flaws are described below –

• Out-of-bounds read in Lighttpd 1.4.45 used in Intel M70KLP series firmware
• Out-of-bounds read in Lighttpd 1.4.35 used in Lenovo BMC firmware
• Out-of-bounds read in Lighttpd before 1.4.51

Intel and Lenovo have opted not to address the issue as the products incorporating the susceptible version of Lighttpd have hit end-of-life (EoL) status and are no longer eligible for security updates, effectively turning it into a forever-day bug.

The disclosure highlights how the presence of outdated third-party components in the latest version of firmware can traverse the supply chain and pose unintended security risks for end users.

“This is yet another vulnerability that will remain unfixed forever in some products and will present high-impact risk to the industry for a very long time,” Binarly added.

Apr 15, 2024NewsroomCloud Security /SaaS Security

The threat actor known as Muddled Libra has been observed actively targeting software-as-a-service (SaaS) applications and cloud service provider (CSP) environments in a bid to exfiltrate sensitive data.

“Organizations often store a variety of data in SaaS applications and use services from CSPs,” Palo Alto Networks Unit 42 said in a report published last week.

“The threat actors have begun attempting to leverage some of this data to assist with their attack progression, and to use for extortion when trying to monetize their work.”

Muddled Libra, also called Starfraud, UNC3944, Scatter Swine, and Scattered Spider, is a notorious cybercriminal group that has leveraged sophisticated social engineering techniques to gain initial access to target networks.

“Scattered Spider threat actors have historically evaded detection on target networks by using living off the land techniques and allowlisted applications to navigate victim networks, as well as frequently modifying their TTPs,” the U.S. government said in an advisory late last year.

The attackers also have a history of monetizing access to victim networks in numerous ways, including extortion enabled by ransomware and data theft.

Unit 42 previously told The Hacker News that the moniker “Muddled Libra” comes from the “confusing muddled landscape” associated with the 0ktapus phishing kit, which has been put to use by other threat actors to stage credential harvesting attacks.

A key aspect of the threat actor’s tactical evolution is the use of reconnaissance techniques to identify administrative users to target when posing as helpdesk staff using phone calls to obtain their passwords.

The recon phase also extends to Muddled Libra, which performs extensive research to find information about the applications and the cloud service providers used by the target organizations.

“The Okta cross-tenant impersonation attacks that occurred from late July to early August 2023, where Muddled Libra bypassed IAM restrictions, display how the group exploits Okta to access SaaS applications and an organization’s various CSP environments,” security researcher Margaret Zimmermann explained.

The information obtained at this stage serves as a stepping stone for conducting lateral movement, abusing the admin credentials to access single sign-on (SSO) portals to gain quick access to SaaS applications and cloud infrastructure.

In the event SSO is not integrated into a target’s CSP, Muddled Libra undertakes broad discovery activities to uncover the CSP credentials, likely stored in unsecured locations, to meet their objectives.

The data stored with SaaS applications are also used to glean specifics about the infected environment, capturing as many credentials as possible to widen the scope of the breach via privilege escalation and lateral movement.

“A large portion of Muddled Libra’s campaigns involve gathering intelligence and data,” Zimmermann said.

“Attackers then use this to generate new vectors for lateral movement within an environment. Organizations store a variety of data within their unique CSP environments, thus making these centralized locations a prime target for Muddled Libra.”

These actions specifically single out Amazon Web Services (AWS) and Microsoft Azure, targeting services like AWS IAM, Amazon Simple Storage Service (S3), AWS Secrets Manager, Azure storage account access keys, Azure Blob Storage, and Azure Files to extract relevant data.

Data exfiltration to an external entity is achieved by abusing legitimate CSP services and features. This encompasses tools like AWS DataSync, AWS Transfer, and a technique called snapshot, the latter of which makes it possible to move data out of an Azure environment by staging the stolen data in a virtual machine.

Muddled Libra’s tactical shift requires organizations to secure their identity portals with robust secondary authentication protections like hardware tokens or biometrics.

“By expanding their tactics to include SaaS applications and cloud environments, the evolution of Muddled Libra’s methodology shows the multidimensionality of cyberattacks in the modern threat landscape,” Zimmermann concluded. “The use of cloud environments to gather large amounts of information and quickly exfiltrate it poses new challenges to defenders.”

Imagine a world where the software that powers your favorite apps, secures your online transactions, and keeps your digital life could be outsmarted and taken over by a cleverly disguised piece of code. This isn’t a plot from the latest cyber-thriller; it’s actually been a reality for years now. How this will change – in a positive or negative direction – as artificial intelligence (AI) takes on a larger role in software development is one of the big uncertainties related to this brave new world.

In an era where AI promises to revolutionize how we live and work, the conversation about its security implications cannot be sidelined. As we increasingly rely on AI for tasks ranging from mundane to mission-critical, the question is no longer just, “Can AI boost cybersecurity?” (sure!), but also “Can AI be hacked?” (yes!), “Can one use AI to hack?” (of course!), and “Will AI produce secure software?” (well…). This thought leadership article is about the latter. Cydrill (a secure coding training company) delves into the complex landscape of AI-produced vulnerabilities, with a special focus on GitHub Copilot, to underscore the imperative of secure coding practices in safeguarding our digital future.

You can test your secure coding skills with this short self-assessment.

The Security Paradox of AI

AI’s leap from academic curiosity to a cornerstone of modern innovation happened rather suddenly. Its applications span a breathtaking array of fields, offering solutions that were once the stuff of science fiction. However, this rapid advancement and adoption has outpaced the development of corresponding security measures, leaving both AI systems and systems created by AI vulnerable to a variety of sophisticated attacks. Déjà vu? The same things happened when software – as such – was taking over many fields of our lives…

At the heart of many AI systems is machine learning, a technology that relies on extensive datasets to “learn” and make decisions. Ironically, the strength of AI – its ability to process and generalize from vast amounts of data – is also its Achilles’ heel. The starting point of “whatever we find on the Internet” may not be the perfect training data; unfortunately, the wisdom of the masses may not be sufficient in this case. Moreover, hackers, armed with the right tools and knowledge, can manipulate this data to trick AI into making erroneous decisions or taking malicious actions.

Copilot in the Crosshairs

GitHub Copilot, powered by OpenAI’s Codex, stands as a testament to the potential of AI in coding. It has been designed to improve productivity by suggesting code snippets and even whole blocks of code. However, multiple studies have highlighted the dangers of fully relying on this technology. It has been demonstrated that a significant portion of code generated by Copilot can contain security flaws, including vulnerabilities to common attacks like SQL injection and buffer overflows.

The “Garbage In, Garbage Out” (GIGO) principle is particularly relevant here. AI models, including Copilot, are trained on existing data, and just like any other Large Language Model, the bulk of this training is unsupervised. If this training data is flawed (which is very possible given that it comes from open-source projects or large Q&A sites like Stack Overflow), the output, including code suggestions, may inherit and propagate these flaws. In the early days of Copilot, a study revealed that approximately 40% of code samples produced by Copilot when asked to complete code based on samples from the CWE Top 25 were vulnerable, underscoring the GIGO principle and the need for heightened security awareness. A larger-scale study in 2023 (Is GitHub’s Copilot as bad as humans at introducing vulnerabilities in code?) had somewhat better results, but still far from good: by removing the vulnerable line of code from real-world vulnerability examples and asking Copilot to complete it, it recreated the vulnerability about 1/3 of the time and fixed the vulnerability only about 1/4 of the time. In addition, it performed very poorly on vulnerabilities related to missing input validation, producing vulnerable code every time. This highlights that generative AI is poorly equipped to deal with malicious input if ‘silver bullet’-like solutions for dealing with a vulnerability (e.g. prepared statements) are not available.

The Road to Secure AI-powered Software Development

Addressing the security challenges posed by AI and tools like Copilot requires a multifaceted approach:

1. Understanding Vulnerabilities: It is essential to recognize that AI-generated code may be susceptible to the same types of attacks as „traditionally” developed software.
2. Elevating Secure Coding Practices: Developers must be trained in secure coding practices, taking into account the nuances of AI-generated code. This involves not just identifying potential vulnerabilities, but also understanding the mechanisms through which AI suggests certain code snippets, to anticipate and mitigate the risks effectively.
3. Adapting the SDLC: It’s not only technology. Processes should also take into account the subtle changes AI will bring in. When it comes to Copilot, code development is usually in focus. But requirements, design, maintenance, testing and operations can also benefit from Large Language Models.
4. Continuous Vigilance and Improvement: AI systems – just as the tools they power – are continually evolving. Keeping pace with this evolution means staying informed about the latest security research, understanding emerging vulnerabilities, and updating the existing security practices accordingly.

Navigating the integration of AI tools like GitHub Copilot into the software development process is risky and requires not only a shift in mindset but also the adoption of robust strategies and technical solutions to mitigate potential vulnerabilities. Here are some practical tips designed to help developers ensure that their use of Copilot and similar AI-driven tools enhances productivity without compromising security.

Implement strict input validation!

Practical Implementation: Defensive programming is always at the core of secure coding. When accepting code suggestions from Copilot, especially for functions handling user input, implement strict input validation measures. Define rules for user input, create an allowlist of allowable characters and data formats, and ensure that inputs are validated before processing. You can also ask Copilot to do this for you; sometimes it actually works well!

Manage dependencies securely!

Practical Implementation: Copilot may suggest adding dependencies to your project, and attackers may use this to implement supply chain attacks via “package hallucination”. Before incorporating any suggested libraries, manually verify their security status by checking for known vulnerabilities in databases like the National Vulnerability Database (NVD) or accomplish a software composition analysis (SCA) with tools like OWASP Dependency-Check or npm audit for Node.js projects. These tools can automatically track and manage dependencies’ security.

Conduct regular security assessments!

Practical Implementation: Regardless of the source of the code, be it AI-generated or hand-crafted, conduct regular code reviews and tests with security in focus. Combine approaches. Test statically (SAST) and dynamically (DAST), do Software Composition Analysis (SCA). Do manual testing and supplement it with automation. But remember to put people over tools: no tool or artificial intelligence can replace natural (human) intelligence.

Be gradual!

Practical Implementation: First, let Copilot write your comments or debug logs – it’s already pretty good in these. Any mistake in these won’t affect the security of your code anyway. Then, once you are familiar with how it works, you can gradually let it generate more and more code snippets for the actual functionality.

Always review what Copilot offers!

Practical Implementation: Never just blindly accept what Copilot suggests. Remember, you are the pilot, it’s “just” the Copilot! You and Copilot can be a very effective team together, but it’s still you who are in charge, so you must know what the expected code is and how the outcome should look like.

Experiment!

Practical Implementation: Try out different things and prompts (in chat mode). Try to ask Copilot to refine the code if you are not happy with what you got. Try to understand how Copilot “thinks” in certain situations and realize its strengths and weaknesses. Moreover, Copilot gets better with time – so experiment continuously!

Stay informed and educated!

Practical Implementation: Continuously educate yourself and your team on the latest security threats and best practices. Follow security blogs, attend webinars and workshops, and participate in forums dedicated to secure coding. Knowledge is a powerful tool in identifying and mitigating potential vulnerabilities in code, AI-generated or not.

Conclusion

The importance of secure coding practices has never been more important as we navigate the uncharted waters of AI-generated code. Tools like GitHub Copilot present significant opportunities for growth and improvement but also particular challenges when it comes to the security of your code. Only by understanding these risks can one successfully reconcile effectiveness with security and keep our infrastructure and data protected. In this journey, Cydrill remains committed to empowering developers with the knowledge and tools needed to build a more secure digital future.

Cydrill’s blended learning journey provides training in proactive and effective secure coding for developers from Fortune 500 companies all over the world. By combining instructor-led training, e-learning, hands-on labs, and gamification, Cydrill provides a novel and effective approach to learning how to code securely.

Check out Cydrill’s secure coding courses.