minsta

Open source machine learning systems are highly vulnerable to security threats

Pabio December 22, 2024

MLflow identified as the most vulnerable open source ML platform
Directory traversal vulnerabilities allow unauthorized access to files in Weave
ZenML Cloud access control issues lead to privilege escalation risks

A recent analysis of the security landscape of machine learning (ML) frameworks found that ML software is prone to more security vulnerabilities than more mature categories like DevOps or web servers.

The growing adoption of machine learning across industries highlights the critical need to secure ML systems, as vulnerabilities can lead to unauthorized access, data breaches, and compromised operations.

THE report from JFrog claims that ML projects such as MLflow have seen an increase in critical vulnerabilities. Over the past few months, JFrog has discovered 22 vulnerabilities in 15 open source ML projects. Among these vulnerabilities, two categories stand out: threats targeting server-side components and risks of privilege escalation within ML frameworks.

Critical vulnerabilities in ML frameworks

The vulnerabilities identified by JFrog affect key components often used in ML workflows, which could allow attackers to exploit tools often recognized by ML practitioners for their flexibility, gain unauthorized access to sensitive files or elevate privileges in ML environments.

One of the vulnerabilities highlighted concerns Weave, a popular toolkit from Weights & Biases (W&B), which helps in tracking and visualizing ML model metrics. The WANDB Weave Directory Traversal vulnerability (CVE-2024-7340) allows low-privileged users to access arbitrary files on the file system.

This flaw is caused by improper input validation when handling file paths, potentially allowing attackers to view sensitive files that could include administrator API keys or other privileged information. Such a breach could lead to privilege escalation, giving attackers unauthorized access to resources and compromising the security of the entire ML pipeline.

ZenML, an MLOps pipeline management tool, is also affected by a critical vulnerability that compromises its access control systems. This flaw allows attackers with minimal access privileges to elevate their permissions within ZenML Cloud, a managed deployment of ZenML, thereby gaining access to restricted information, including confidential secrets or model files.

The access control issue in ZenML exposes the system to significant risk, as elevated privileges could allow an attacker to manipulate ML pipelines, tamper with model data, or access sensitive operational data, thereby could impact production environments dependent on these pipelines.

Another serious vulnerability, known as Deep Lake Command Injection (CVE-2024-6507), was discovered in Deep Lake Database, a data storage solution optimized for AI applications. This vulnerability allows attackers to execute arbitrary commands by exploiting how Deep Lake handles imports of external datasets.

Due to improper command verification, an attacker could potentially execute code remotely, compromising the security of the database and any connected applications.

A notable vulnerability was also discovered in Vanna AI, a tool designed for generating and visualizing natural language SQL queries. Vanna.AI Prompt Injection (CVE-2024-5565) allows attackers to inject malicious code into SQL prompts, which the tool then processes. This vulnerability, which could lead to remote code execution, allows malicious actors to target Vanna AI’s SQL to Graph visualization functionality to manipulate visualizations, execute SQL injections, or exfiltrate data.

Mage.AI, an MLOps tool for managing data pipelines, was found to have multiple vulnerabilities, including unauthorized shell access, arbitrary file leaks, and weak path traversal checks.

These issues allow attackers to take control of data pipelines, expose sensitive configurations, or even execute malicious commands. The combination of these vulnerabilities poses a high risk of privilege escalation and data integrity violation, compromising the security and stability of ML pipelines.

By gaining administrative access to databases or ML registries, attackers can embed malicious code into models, leading to backdoors that activate when the model is loaded. This can compromise downstream processes as models are used by various teams and CI/CD pipelines. Attackers can also exfiltrate sensitive data or conduct model poisoning attacks to degrade model performance or manipulate outputs.

JFrog’s findings highlight an operational gap in MLOps security. Many organizations lack strong integration of AI/ML security practices with broader cybersecurity strategies, leaving potential blind spots. As ML and AI continue to drive significant advancements in the industry, protecting the frameworks, data sets, and models that power these innovations becomes paramount.

Mondor Festival

Mondor Festival

Open source machine learning systems are highly vulnerable to security threats

Pabio

Vacant home on Winnipeg’s Redwood Avenue sees fourth fire in less than a year

Work underway to create domestic violence court – The Royal Gazette

Hastings Art Gallery Chronicle: Scouring the length and breadth of the country to collect key works of art

Snakebite deaths may be rare, but would you know what to do if you are bitten?

Open source machine learning systems are highly vulnerable to security threats

Pabio

You Might Also Like

Vacant home on Winnipeg’s Redwood Avenue sees fourth fire in less than a year

Work underway to create domestic violence court – The Royal Gazette

Hastings Art Gallery Chronicle: Scouring the length and breadth of the country to collect key works of art

Snakebite deaths may be rare, but would you know what to do if you are bitten?