Mar 15, 2024NewsroomData Privacy / Artificial Intelligence

Cybersecurity researchers have found that third-party plugins available for OpenAI ChatGPT could act as a new attack surface for threat actors looking to gain unauthorized access to sensitive data.

According to new research published by Salt Labs, security flaws found directly in ChatGPT and within the ecosystem could allow attackers to install malicious plugins without users’ consent and hijack accounts on third-party websites like GitHub.

ChatGPT plugins, as the name implies, are tools designed to run on top of the large language model (LLM) with the aim of accessing up-to-date information, running computations, or accessing third-party services.

OpenAI has since also introduced GPTs, which are bespoke versions of ChatGPT tailored for specific use cases, while reducing third-party service dependencies. As of March 19, 2024, ChatGPT users will no longer be able to install new plugins or create new conversations with existing plugins.

One of the flaws unearthed by Salt Labs involves exploiting the OAuth workflow to trick a user into installing an arbitrary plugin by taking advantage of the fact that ChatGPT doesn’t validate that the user indeed started the plugin installation.

This effectively could allow threat actors to intercept and exfiltrate all data shared by the victim, which may contain proprietary information.

The cybersecurity firm also unearthed issues with PluginLab that could be weaponized by threat actors to conduct zero-click account takeover attacks, allowing them to gain control of an organization’s account on third-party websites like GitHub and access their source code repositories.

“‘auth.pluginlab[.]ai/oauth/authorized’ does not authenticate the request, which means that the attacker can insert another memberId (aka the victim) and get a code that represents the victim,” security researcher Aviad Carmel explained. “With that code, he can use ChatGPT and access the GitHub of the victim.”

The memberId of the victim can be obtained by querying the endpoint “auth.pluginlab[.]ai/members/requestMagicEmailCode.” There is no evidence that any user data has been compromised using the flaw.

Also discovered in several plugins, including Kesem AI, is an OAuth redirection manipulation bug that could permit an attacker to steal the account credentials associated with the plugin itself by sending a specially crafted link to the victim.

The development comes weeks after Imperva detailed two cross-site scripting (XSS) vulnerabilities in ChatGPT that could be chained to seize control of any account.

In December 2023, security researcher Johann Rehberger demonstrated how malicious actors could create custom GPTs that can phish for user credentials and transmit the stolen data to an external server.

New Remote Keylogging Attack on AI Assistants

The findings also follow new research published this week about an LLM side-channel attack that employs token-length as a covert means to extract encrypted responses from AI Assistants over the web.

“LLMs generate and send responses as a series of tokens (akin to words), with each token transmitted from the server to the user as it is generated,” a group of academics from the Ben-Gurion University and Offensive AI Research Lab said.

“While this process is encrypted, the sequential token transmission exposes a new side-channel: the token-length side-channel. Despite encryption, the size of the packets can reveal the length of the tokens, potentially allowing attackers on the network to infer sensitive and confidential information shared in private AI assistant conversations.”

This is accomplished by means of a token inference attack that’s designed to decipher responses in encrypted traffic by training an LLM model capable of translating token-length sequences into their natural language sentential counterparts (i.e., plaintext).

In other words, the core idea is to intercept the real-time chat responses with an LLM provider, use the network packet headers to infer the length of each token, extract and parse text segments, and leverage the custom LLM to infer the response.

Two key prerequisites to pulling off the attack are an AI chat client running in streaming mode and an adversary who is capable of capturing network traffic between the client and the AI chatbot.

To counteract the effectiveness of the side-channel attack, it’s recommended that companies that develop AI assistants apply random padding to obscure the actual length of tokens, transmit tokens in larger groups rather than individually, and send complete responses at once, instead of in a token-by-token fashion.

“Balancing security with usability and performance presents a complex challenge that requires careful consideration,” the researchers concluded.

Mar 05, 2024NewsroomVulnerability / Network Security

A new pair of security vulnerabilities have been disclosed in JetBrains TeamCity On-Premises software that could be exploited by a threat actor to take control of affected systems.

The flaws, tracked as CVE-2024-27198 (CVSS score: 9.8) and CVE-2024-27199 (CVSS score: 7.3), have been addressed in version 2023.11.4. They impact all TeamCity On-Premises versions through 2023.11.3.

“The vulnerabilities may enable an unauthenticated attacker with HTTP(S) access to a TeamCity server to bypass authentication checks and gain administrative control of that TeamCity server,” JetBrains said in an advisory released Monday.

TeamCity Cloud instances have already been patched against the two flaws. Cybersecurity firm Rapid7, which discovered and reported the issues on February 20, 2024, said CVE-2024-27198 is a case of authentication bypass that allows for a complete compromise of a susceptible server by a remote unauthenticated attacker.

“Compromising a TeamCity server allows an attacker full control over all TeamCity projects, builds, agents and artifacts, and as such is a suitable vector to position an attacker to perform a supply chain attack,” the company noted.

CVE-2024-27199, also an authentication bypass flaw, stems from a path traversal issue that can permit an unauthenticated attacker to replace the HTTPS certificate in a vulnerable TeamCity server with a certificate of their choosing via the “/app/https/settings/uploadCertificate” endpoint and even alter the port number the HTTPS service listens on.

A threat actor could leverage the vulnerability to perform a denial-of-service against the TeamCity server by either changing the HTTPS port number, or by uploading a certificate that will fail client-side validation. Alternatively, the uploaded certificate could be used for adversary-in-the-middle scenarios if it’s trusted by the clients.

“This authentication bypass allows for a limited number of authenticated endpoints to be reached without authentication,” Rapid7 said of the shortcoming.

“An unauthenticated attacker can leverage this vulnerability to both modify a limited number of system settings on the server, as well as disclose a limited amount of sensitive information from the server.”

The development comes nearly a month after JetBrains released fixes to contain another flaw (CVE-2024-23917, CVSS score: 9.8) that could also enable an unauthenticated attacker to gain administrative control of TeamCity servers.

With security vulnerabilities in JetBrains TeamCity having come under active exploitation last year by North Korean and Russian threat actors, it’s essential that users take steps to update their servers immediately.