Mar 15, 2024NewsroomData Privacy / Artificial Intelligence

Cybersecurity researchers have found that third-party plugins available for OpenAI ChatGPT could act as a new attack surface for threat actors looking to gain unauthorized access to sensitive data.

According to new research published by Salt Labs, security flaws found directly in ChatGPT and within the ecosystem could allow attackers to install malicious plugins without users’ consent and hijack accounts on third-party websites like GitHub.

ChatGPT plugins, as the name implies, are tools designed to run on top of the large language model (LLM) with the aim of accessing up-to-date information, running computations, or accessing third-party services.

OpenAI has since also introduced GPTs, which are bespoke versions of ChatGPT tailored for specific use cases, while reducing third-party service dependencies. As of March 19, 2024, ChatGPT users will no longer be able to install new plugins or create new conversations with existing plugins.

One of the flaws unearthed by Salt Labs involves exploiting the OAuth workflow to trick a user into installing an arbitrary plugin by taking advantage of the fact that ChatGPT doesn’t validate that the user indeed started the plugin installation.

This effectively could allow threat actors to intercept and exfiltrate all data shared by the victim, which may contain proprietary information.

The cybersecurity firm also unearthed issues with PluginLab that could be weaponized by threat actors to conduct zero-click account takeover attacks, allowing them to gain control of an organization’s account on third-party websites like GitHub and access their source code repositories.

“‘auth.pluginlab[.]ai/oauth/authorized’ does not authenticate the request, which means that the attacker can insert another memberId (aka the victim) and get a code that represents the victim,” security researcher Aviad Carmel explained. “With that code, he can use ChatGPT and access the GitHub of the victim.”

The memberId of the victim can be obtained by querying the endpoint “auth.pluginlab[.]ai/members/requestMagicEmailCode.” There is no evidence that any user data has been compromised using the flaw.

Also discovered in several plugins, including Kesem AI, is an OAuth redirection manipulation bug that could permit an attacker to steal the account credentials associated with the plugin itself by sending a specially crafted link to the victim.

The development comes weeks after Imperva detailed two cross-site scripting (XSS) vulnerabilities in ChatGPT that could be chained to seize control of any account.

In December 2023, security researcher Johann Rehberger demonstrated how malicious actors could create custom GPTs that can phish for user credentials and transmit the stolen data to an external server.

New Remote Keylogging Attack on AI Assistants

The findings also follow new research published this week about an LLM side-channel attack that employs token-length as a covert means to extract encrypted responses from AI Assistants over the web.

“LLMs generate and send responses as a series of tokens (akin to words), with each token transmitted from the server to the user as it is generated,” a group of academics from the Ben-Gurion University and Offensive AI Research Lab said.

“While this process is encrypted, the sequential token transmission exposes a new side-channel: the token-length side-channel. Despite encryption, the size of the packets can reveal the length of the tokens, potentially allowing attackers on the network to infer sensitive and confidential information shared in private AI assistant conversations.”

This is accomplished by means of a token inference attack that’s designed to decipher responses in encrypted traffic by training an LLM model capable of translating token-length sequences into their natural language sentential counterparts (i.e., plaintext).

In other words, the core idea is to intercept the real-time chat responses with an LLM provider, use the network packet headers to infer the length of each token, extract and parse text segments, and leverage the custom LLM to infer the response.

Two key prerequisites to pulling off the attack are an AI chat client running in streaming mode and an adversary who is capable of capturing network traffic between the client and the AI chatbot.

To counteract the effectiveness of the side-channel attack, it’s recommended that companies that develop AI assistants apply random padding to obscure the actual length of tokens, transmit tokens in larger groups rather than individually, and send complete responses at once, instead of in a token-by-token fashion.

“Balancing security with usability and performance presents a complex challenge that requires careful consideration,” the researchers concluded.

Dec 15, 2023NewsroomPrivacy / User Tracking

Google on Thursday announced that it will start testing a new feature called “Tracking Protection” beginning January 4, 2024, to 1% of Chrome users as part of its efforts to deprecate third-party cookies in the web browser.

The setting is designed to limit “cross-site tracking by restricting website access to third-party cookies by default,” Anthony Chavez, vice president of Privacy Sandbox at Google, said.

The tech giant noted that participants for Tracking Protection will be selected at random and that chosen users will be notified upon opening Chrome on either a desktop or an Android device.

The goal is to restrict third-party cookies (also called “non-essential cookies”) by default, preventing them from being used to track users as they move from one website to the other for serving personalized ads.

While several major browsers like Apple Safari and Mozilla Firefox have either already placed restrictions on third-party cookies via features like Intelligent Tracking Prevention (ITP) and Enhanced Tracking Protection in Firefox, Google is taking more of a middle-ground approach that involves devising alternatives where users can access free online content and services without compromising on their privacy.

In mid-October 2023, Google confirmed its plans to “disable third-party cookies for 1% of users from Q1 2024 to facilitate testing, and then ramp up to 100% of users from Q3 2024.”

Privacy Sandbox, instead of providing a cross-site or cross-app user identifier, “aggregates, limits, or noises data” through APIs like Protected Audience (formerly FLEDGE), Topics, and Attribution Reporting to help prevent user re-identification.

In doing so, the goal is to block third-parties from tracking user browsing behavior across sites, while still allowing sites and apps to serve relevant ads and enabling advertisers to measure the performance of their online ads without using individual identifiers.

“With Tracking Protection, Privacy Sandbox and all of the features we launch in Chrome, we’ll continue to work to create a web that’s more private than ever, and universally accessible to everyone,” Chavez said.