Developers love details. It’s the details and technicalities of developing software applications, coupled with the fundamental need to appreciate algorithmic logic, that keeps engineers coding into the wee hours of the morning, when in reality they should have left their office and go home for a bowl of instant noodles.

As developers create and “cut” code, they naturally leave a trail of annotations and so-called artifacts along the way, ranging from notes on user requirements to design documentation and more. by use cases, reference architecture documents, etc. These fingerprints are acceptable…but they may represent sensitive data elements in some cases. More importantly, developers will inevitably leave a paper trail of credentials, application programming interface tokens, and passwords embedded in their code, which can remain exposed for years, even after they are removed.

That’s a concern, warns Yadir Kadkoda, principal security researcher at Aqua Security, a company known for its work in cloud-native security controls covering containerized and serverless workloads.

Search Aqua Nautilus suggests that up to 18% of “software secrets” are inadvertently exposed and not proactively discovered, meaning there could be more to discover.

Schrödinger’s software

“Everyone knows Schrödinger’s cat of quantum mechanics: both alive and dead simultaneously until the box is opened and the cat is observed. This quantum paradox reflects a critical risk in modern software development: secrets embedded in code,” Kadkoda said. “The problem stems from a fundamental design flaw in the Git-based infrastructure (the popular code repository and version control system used by developers). Since this architecture underlies most source code management systems, including GitHub, GitLab, and Bitbucket, it affects almost all popular DevOps platforms. Our research revealed a large number of secrets belonging to Fortune 500 companies only on GitHub.

To address this problem, historical secrets analysis is proposed as a way to proactively identify and eliminate these phantom secrets before they fall into the wrong hands. But Kadkoda called these things secrets, so what does that mean? Secrets are, by nature, information intended to be confidential, such as financial information, passwords, intellectual property, etc. These valuable assets must be protected from unauthorized access by malicious actors who wish to exploit them for their own gain or to harm a person or company.

“In the context of coding, these secrets may consist of API keys, identifiers, passwords or telemetry platforms that could be vulnerable through insecure code, leaked code repositories or channel channels. unencrypted communication. When cybercriminals discover a secret, it allows them more access to systems and data, leading to larger security breaches,” Kadkoda explained. “This is why, for years, organizations have been advised not to hard-code secrets into their code. However, doing it just once can reveal a secret permanently, even if it was deleted after the fact. By performing a thorough secrets analysis, organizations can expect various positive results.

Ex-ante beats post-ante

The benefits of this covert scanning include the ability to prevent breaches by finding and remediating potential threats before they can be exploited. Because most industries comply with various regulatory frameworks with strict data protection requirements; digitalization naturally improves this compliance. There are also opportunities to avoid reputational damage resulting from breaches and, of course, to avoid the cost of remediation procedures if software systems end up cracking.

From a software application development perspective, it’s all about making the right choice ex ante rather than post ante.

“Historical Secrets Analysis works by analyzing coder validations and other data sources to find sensitive information using a range of tools and technologies,” Kadkoda detailed. “Expression-based analysis searching for patterns and keywords related to secrets using regular expressions (i.e. an expression in the sense of the word in software code being a sequence of characters that forms a pattern in the text that can be matched with another). These expressions can be customized to match specific secret types, such as passwords or APIs, and can be applied to several different types of data sources.

Other techniques detailed here by the Aqua team include dictionary-based scanning to use predefined dictionaries of known secrets to search for vulnerabilities in multiple data sources, such as log files and configuration files. Machine learning analytics involves using algorithms to analyze data for patterns associated with secrets. By training algorithms on large data sets, businesses can more easily find hidden patterns and anomalies. Hybrid analysis involves combining the above approaches to improve the accuracy and coverage of secrets analysis and identify more complex types of secrets.

Empowering DevSecOps teams

“The right automated tools will enable comprehensive analysis of an organization’s IT infrastructure, allowing DevSecOps teams to focus on more strategic activities. However, when it comes to choosing the right tool, the DevSecOps professional must consider several things,” said Kadkoda. “Tools should be intuitive and easy to use, with clear feedback and guidance on how to resolve vulnerabilities – and they should also integrate seamlessly into existing workflows and processes. By integrating these practices into the IT team’s continuous integration and deployment CD pipeline, the tools will enable automation of code analysis and other artifacts.

Here we see that it is important to cover code, container images, Kubernetes configurations and other areas of the software development stack, so that vulnerabilities can be identified and resolved at all stages of the development process . Using machine learning and AI can reduce the occurrence of false positives and negatives.

“Tools at this level should be created with code privacy in mind and should never expose code to external parties. All data collected must be stored securely and in accordance with any applicable data protection regulations. Somewhere buried within a company’s infrastructure, there may be many dusty relics just waiting for the right (or wrong!) person to exploit them. Now is the time to equip DevOps teams with the right tools to perform secret analysis and continually bring them to light,” said Kadkoda.

There is a certain amount of unknown here, which is why there is so much talk about the search for invisible secrets that must be brought to light. While it’s not really about chasing ghosts, it is certainly a process of unmasking and exposing those “nasty” elements inside the software stack before they can be used for nefarious purposes.