Fear and Loathing DevSecOps
We had 2 code analyzers, 4 tools for dynamic testing, our own crafts and 250 scripts. Not that it was all necessary in the current process, but once I started to implement DevSecOps, then you have to go to the end.
Source of Characters: Justin Royland and Dan Harmon.
')
What is SecDevOps? And DevSecOps? What are the differences? Application Security - what is it about? Why does the classic approach no longer work? Yury Shabalin from Swordfish Security knows the answer to all these questions . Yuri will answer and explain in detail everything about the problems of transition from the classical Application Security model to the DevSecOps process: how to approach the embedding of a secure development process into the DevOps process and not break anything, how to go through the main stages of security testing, what tools you can apply, than they differ and how to adjust them correctly to avoid pitfalls.
About Speaker: Yuri Shabalin - Chief Security Architect at Swordfish Security . Responsible for the implementation of SSDL, for the overall integration of application analysis tools into a single ecosystem of development and testing. 7 years experience in information security. He worked at Alfa-Bank, Sberbank and Positive Technologies, which develops software and provides services. Speaker of international conferences ZerONights, PHDays, RISSPA, OWASP.
Application Security is a security section that is responsible for application security. This does not apply to the infrastructure or network security, namely, what we write and what the developers are working on - these are the shortcomings and vulnerabilities of the application itself.
The direction of SDL or SDLC - Security development lifecycle - was developed by Microsoft. The diagram shows the canonical SDLC model, the main task of which is to take part in security at every stage of development, from requirements to release and release to production. Microsoft realized that there are too many bugs in the prom, there are more of them and we need to do something with it, and suggested this approach, which has become canonical.
Application Security and SSDL are not aimed at detecting vulnerabilities, as is commonly believed, but to prevent their occurrence. Over time, the canonical approach from Microsoft was improved, developed, and a deeper, detailed immersion appeared.
Canonical SDLC is highly detailed in various methodologies - OpenSAMM, BSIMM, OWASP. Methodologies are different, but, in general, are similar.
I prefer BSIMM most of all - Building Security In Maturity Model . The basis of the methodology is the division of the Application Security process into 4 domains: Governance, Intelligence, SSDL Touchpoints and Deployment. In each domain there are 12 practices, which are presented in the form of 112 activities.
Each of the 112 activities has 3 levels of maturity : beginner, intermediate and advanced. All 12 practices can be studied by sections, select important things for you, understand how to implement them and gradually add elements, for example, static and dynamic code analysis or code review. You paint the plan and work quietly on it as part of implementing the selected activities.
DevOps initially assumed security checks. In practice, the number of security teams was much smaller than they are now, and they did not act as participants in the process, but as a control and supervisory body that makes demands on it and checks the quality of the product at the end of the release. This is a classic approach in which security teams were behind the wall from development and did not take part in the process.
The main problem is that the information security is separate from the development. This is usually a kind of IB circuit and there are 2-3 large and expensive tools in it. Every six months, the source code or application arrives that needs to be checked, and pentests are made once a year. This all leads to the fact that the timing of the release in the industry is postponed, and a huge number of vulnerabilities from automated tools fall out on the developer. All this can not be disassembled and repaired, because even in the previous six months the results were not dismantled, and here a new batch.
In the course of our company's work, we see that safety in all areas and industries understands that it is time to catch up and spin with the development in one wheel - in Agile . The DevSecOps paradigm fits nicely on the methodology of agile development, on implementation, support and participation in each release and iteration.
The most important word in the Security Development Lifecycle is “process . ” You need to understand this before thinking about buying tools.
Often planning for a secure development process begins with the selection and purchase of a tool, and ends with attempts to integrate the tool into the current process, which remain attempts. This leads to sad consequences, because all the tools have their own characteristics and limitations.
A common case is when the security department chose a good, expensive tool with ample opportunities, and came to the developers to embed it in the process. But it does not come out - the process is structured so that the limitations of the instrument already purchased do not fall into the current paradigm.
Before buying expensive tools, look at what you already have. In each company there are safety requirements, which are presented to the development, there are checks, pentests - why not transform all this into a clear and convenient form for everyone?
Usually the requirements - it is a paper talmud, which lies on the shelf. There was a case when we come to the company to see the processes and ask to show the security requirements for the software. The specialist who did this was looking for a long time:
- Now, somewhere in the notes there was a way where this document lies.
As a result, we received the document a week later.
For requirements, checks and other things, create a page, for example, on Confluence - this is convenient for everyone.
Usually, in an average company, one security officer works for 100-200 developers, who performs several functions and does not physically have time to check everything. Even if he tries his best - he alone will not check all the code that the development generates. For such cases, the concept has been developed - Security Champions .
Security Champion is the entry point to the development team and security evangelist in one person.
Usually, when the security team comes to the development team and indicates an error in the code, he gets a surprised answer:
- And who are you? I see you for the first time. Everything is good with me - I put a “apply” on my senior comrade on code review, we move on!
This is a typical situation, because the elders or just teammates with whom the developer constantly interacts in work and in code review are much more trustworthy. If, instead of the security officer, the Security Champion indicates an error and its consequences, then his word will have more weight.
Also, developers know their code better than any security specialist. For a person who has at least 5 projects in a static analysis tool, it is usually difficult to remember all the nuances. Security Champions know their product: what interacts with what and what to look at first - they are more effective.
So think about embedding Security Champions and expanding the influence of the security team. It is also useful for the champion himself: professional development in the new field, expansion of the technical outlook, improvement of technical, managerial and leadership skills, increase in market value. This is some element of social engineering, your “eyes” in the development team.
A paradigm of 20 on 80 says that 20% of the effort gives 80% of the result. These 20% are application analysis practices that can and should be automated. Examples of such activities are static analysis — SAST , dynamic analysis — DAST, and Open Source control . I'll tell you more about the activity, as well as about the tools, what features we usually encounter when they are introduced into the process, and how to do it correctly.
I will highlight issues that are relevant to all tools that need attention. I will analyze them in more detail, so as not to repeat further.
Long time analysis. If it takes 30 minutes to complete the tests and assembly, from the commit to the exit to the prod, the information security checks will take a day. So no one will slow down the process. Consider this feature and draw conclusions.
False Negative High or False Positive. All products are different, all use different frameworks and their own style of writing code. On different code bases and technologies, tools can show different levels of False Negative and False Positive. Therefore, look what exactly in your company and for your applications will show a good and reliable result.
No integration with existing tools . Look at the tools in terms of integrations, so that you already use. For example, if you have Jenkins or TeamCity, check the integration of the tools with this particular software, and not with GitLab CI, which you are not using.
Absence or excessive complexity of customization. If the tool has no API, then why is it needed? Everything that can be done in the interface should be accessible through the API. Ideally, the tool should have the ability to customize the checks.
No roadmap of product development. Development does not stand still, we always use new frameworks and functions, we rewrite the old code into new languages. We want to be sure that the tool we buy will support new frameworks and technologies. Therefore, it is important to know that the product has a real and proper Roadmap development.
In addition to the features of the tools, consider the features of the development process. For example, interfere with the development - this is a typical error. Let's see what other features should be considered and what the security team should pay attention to.
In order not to disrupt development and release dates, create different rules and different show stoppers — criteria for stopping the build process in the presence of vulnerabilities — for different environments . For example, we understand that the current branch goes to the developer booth or UAT, which means we do not stop and do not say:
- You have vulnerabilities here, you will not go anywhere further!
At this stage, it is important to tell the developers that there are security issues that are worth paying attention to.
The presence of vulnerabilities is not an obstacle for further testing : manual, integration or manual. On the other hand, we need to somehow improve the safety of the product, and that the developers do not forget about what it finds safe. Therefore, sometimes we do this: on the stand, when rolling out to the development environment, we simply notify the development:
- Guys, you have problems, please pay attention to them.
At the UAT stage, we again show warnings about vulnerabilities, and at the stage of exit in the prom we say:
- Guys, we warned several times that you didn’t do anything - we won’t let you go with that.
If we talk about code and dynamics, then it is necessary to show and warn about vulnerabilities only those features and code that was just written in this feature. If the developer moved the button by 3 pixels and we tell him that he has a SQL injection there and therefore it is urgently necessary to correct it - this is wrong. Look only at what is written now, and at the change that comes in the application.
Suppose we have a certain functional defect - the way the application should not work: money is not transferred, when you click on the button there is no transition to the next page or the product is not loaded. Security defects - these are the same defects, but not in the context of the application, and security.
Since all vulnerabilities are the same defects, they should be located in the same place as all development defects. So forget about reports and scary pdfs that nobody reads.
When I worked in a company that was developing, I received a report from the static analysis tools. I opened it, was terrified, made coffee, looked through 350 pages, closed it and went on to work further. Big reports are dead reports . Usually they do not go anywhere, letters are deleted, forgotten, lost, or the business says that it takes risks.
What to do? Confirmed defects that have been found are simply converted into a form suitable for development, for example, we add them to the backlog in Jira. We prioritize defects and eliminate them in order of priority, along with functional defects and defects of tests.
This is an analysis of the code for the presence of vulnerabilities , but it is not the same as SonarQube. We check not only by patterns or style. In the analysis, a number of approaches are used: on the vulnerability tree, on DataFlow , on the analysis of configuration files. This is all that concerns the code itself.
Advantages of the approach : identification of vulnerabilities in the code at an early stage of development , when there are no stands and a finished tool, and the possibility of incremental scanning : scanning a section of code that has changed, and only the feature that we are doing now, which reduces the scan time.
The disadvantages are the lack of support for the required languages.
The necessary integrations that should be in the tools, in my subjective opinion:
The picture shows some of the best representatives of static analysis.
Not the tools are important, but the process, so there are Open Source solutions that are also good for running the process.
SAST Open Source will not find a huge number of vulnerabilities or complex DataFlow, but you can and should use them when building the process. They help to understand how the process will be built, who will respond to bugs, who will report, who - to report. If you want to conduct the initial step of building the security of your code - use the Open Source solution.
How can this be integrated if you are at the beginning of the journey, you have nothing: neither CI, nor Jenkins, nor TeamCity? Consider integration into the process.
If you have Bitbucket or GitLab, you can do the integration at the level of Concurrent Versions System .
On event - pull request, commit. You scan the code and, in the build status, show that the security check has passed or failed.
Feedback. Of course, feedback is always needed. If you just performed on the security side, put it in a box and did not tell anyone about it, and then at the end of the month dumped a bunch of bugs - this is not right and not good.
Once, we set up a number of important projects as the default reviewer of the technical user AppSec. Depending on whether errors are detected in the new code or there are no errors, at pull request, the reviewer puts the status on “accept” or “need work” - either everything is OK, or you need to refine and refer to what exactly to refine. For integration with the version that goes to the prod, we had a ban on merge if the test on IB is not passed. We included this in the manual code review, and the rest of the process saw the security statuses for this particular process.
Many have a quality gate for code quality. Here is the same thing - you can do the same gates only for SAST tools. There will be the same interface, the same quality gate, only it will be called the security gate . And also, if you have a process using the SonarQube, you can easily integrate everything there.
Here, too, everything is quite simple:
This is all in a perfect pink world. In real life, this is not, but we strive. The result of performing security checks should be similar to the results of unit tests.
For example, we took a big project and decided that now we will scan it with SAST, OK. We crammed this project into SAST, it gave us 20,000 vulnerabilities, and we decided by good will that everything was fine. 20,000 vulnerabilities are our technical debt. We will put the debt in the box, we will quietly rake and get bugs into the defect-trackers. We will hire a company, do it all ourselves or Security Champions will help us - and technical debt will decrease.
And all newly emerging vulnerabilities in the new code should be eliminated as well as errors in the unit or in the autotests. Relatively speaking, the assembly started, drove out, two tests and two safety tests fell. OK - we went, looked at what happened, corrected one, corrected the second, the next time they drove away - everything is fine, no new vulnerabilities have appeared, the tests are not failed. If this task is deeper and you need to understand it well, or fixing vulnerabilities affects large layers of what lies under the hood: you have introduced a bug in the defect tracker, it is prioritized and corrected. Unfortunately, the world is not perfect and tests sometimes fall.
An example security gate is an analogue of the quality gate, in terms of the presence and number of vulnerabilities in the code.
We integrate with SonarQube - the plugin is installed, everything is very convenient and cool.
Integration Capabilities:
Something like this is getting the results from the server.
In our Intellij IDEA development environment, an additional item simply appears, which reports that such vulnerabilities were discovered during the scan. You can immediately edit the code, watch recommendations and Flow Graph . This is all located in the developer’s workplace, which is very convenient - you don’t need to follow the other links and watch something extra.
This is my favorite topic. Everyone uses the Open Source library - why write a bunch of crutches and bicycles when you can get a ready-made library in which everything is already implemented?
Of course, this is true, but libraries are also written by people, also include certain risks and also there are vulnerabilities, which are periodically or permanently reported. Therefore, there is the next step in Application Security - an analysis of the Open Source component.
The tool includes three large stages.
Search for vulnerabilities in libraries. For example, the tool knows that we are using some kind of library, and that there are some vulnerabilities in CVE or in bug trackers that relate to this version of the library. If you try to use it, the tool will warn you that the library is vulnerable and advises using a different version where there are no vulnerabilities.
Analysis of license purity. With us this is not very popular yet, but if you work abroad, you can periodically get an atat for using open source components that cannot be used or modified. According to the policy of the licensed library, we cannot do this. Or, if we modified it and use it, then we have to lay out our code. Of course, no one wants to upload the code of their products, but you can protect against this too.
Analysis of the components that are used in the industrial environment. Imagine a hypothetical situation that we finally completed the development and released the last release of our microservice in the prom. He lives there remarkably - a week, a month, a year. We do not collect it, we do not conduct security checks, everything seems to be fine. But suddenly, two weeks after the release, there is a critical vulnerability in the Open Source component, which we use precisely in this assembly, in the industrial environment. If we do not write down what and where we use, then this vulnerability will simply not be seen. Some tools have the ability to monitor vulnerabilities in libraries that are now used in prom. It is very useful.
Opportunities:
A few examples of area leaders who are engaged in the analysis of Open Source.
The only free one is OWASP's Dependency-Check . You can turn it on at the first stages, see how it works and what it supports. Basically, these are all cloud products, or on-premise, but behind their base they still go to the Internet. They do not send your libraries, but hashes or their values, which they calculate, and fingerprints to their server to receive news of the presence of vulnerabilities.
Control perimeter libraries that are downloaded from external sources. We have external and internal repositories. For example, inside Event Central stands Nexus, and we want to have no “critical” or “high” status vulnerabilities inside our repository. You can configure proxying using the Nexus Firewall Lifecycle tool so that such vulnerabilities are cut off and do not fall into the internal repository.
Integration into CI . On the same level as autotests, unit tests and development stages: dev, test, prod. At each stage, you can download any library, use anything, but if there is something hard with a status of "critical", it is possible, you should pay attention to this developers at the stage of prom.
Integration with artifacts : Nexus and JFrog.
Integration into the development environment. The tools you choose should be integrated with development environments. The developer should have access to the scan results from his workplace, or be able to scan himself and check the code for vulnerabilities before committing to CVS.
CD integration. This is a cool feature that I really like and about which I have already said - monitoring the emergence of new vulnerabilities in the industrial environment. It works like this.
We have Public Component Repositories - some tools outside, and our internal repository. We want to have only trusted components. When proxying a request, we check that the library being downloaded does not have vulnerabilities. If it falls under certain policies that we set and necessarily coordinate with the development, then we don’t download it, and the beating comes to use another version. Accordingly, if there is something really critical and bad in the library, then the developer at the installation stage will not receive the library - let him use the version above or below.
The tools of dynamic analysis are fundamentally different from everything that was said before. This is a kind of imitation of user experience with the application. If this is a web application, we send requests, imitating the work of the client, click on the buttons on the front, send artificial data from the form: quotes, brackets, characters in different encoding, to look at how the application works and processes external data.
The same system allows you to check template vulnerabilities in Open Source. Since DAST does not know which Open Source we are using, it simply throws “malicious” patterns and analyzes the server's responses:
- Yeah, there is a problem of deserialization, but not here.
There are big risks in this, because if you run this security test on the same stand with which testers work, unpleasant things can happen.
We had a situation when we finally launched AppScan: we knocked out access to the application for a long time, got 3 accounts and were delighted - finally we’ll check everything! They started the scan, and the first thing that AppScan did was to get into the admin panel, poke through all the buttons, change half of the data, and then kill the server with their own mailform queries. Development with testing said:
- Guys, are you kidding me ?! We gave you accounts, and you put a stand!
Consider the possible risks. Ideally, prepare a separate stand for testing the IB, which will be isolated from the rest of the environment at least somehow, and conditionally check the admin panel preferably in manual mode. This pentest is the remaining percentage of effort that we are not currently considering.
It is worth considering that you can use it as an analogue of load testing. At the first stage, you can turn on the dynamic scanner in 10-15 threads and see what happens, but usually, as practice shows, nothing good.
Several resources that are commonly used.
The Burp Suite is a “Swiss knife” for any security professional. It is used by everyone, and it is very convenient. - enterprise edition. stand alone , - , . , .
: .
mock-, — , .
, . — , , OpenSource, - , , Waf .
, , , , -.
. , , . , , . , .
API, , , , — AppSec , .
, security- , , , , , . , , — Confluence , Jira -, / CI/CD.
. — . , , . — «» , — - high mega super critical, , .
— , . , , . DevSecOps, SecDevOps, .
, : , , , , . — . — .
.
, . , — . - , . , .
— Security Champions .
, , , - — . , . , community, , . , .
.
Source of Characters: Justin Royland and Dan Harmon.
')
What is SecDevOps? And DevSecOps? What are the differences? Application Security - what is it about? Why does the classic approach no longer work? Yury Shabalin from Swordfish Security knows the answer to all these questions . Yuri will answer and explain in detail everything about the problems of transition from the classical Application Security model to the DevSecOps process: how to approach the embedding of a secure development process into the DevOps process and not break anything, how to go through the main stages of security testing, what tools you can apply, than they differ and how to adjust them correctly to avoid pitfalls.
About Speaker: Yuri Shabalin - Chief Security Architect at Swordfish Security . Responsible for the implementation of SSDL, for the overall integration of application analysis tools into a single ecosystem of development and testing. 7 years experience in information security. He worked at Alfa-Bank, Sberbank and Positive Technologies, which develops software and provides services. Speaker of international conferences ZerONights, PHDays, RISSPA, OWASP.
Application Security: what is it about?
Application Security is a security section that is responsible for application security. This does not apply to the infrastructure or network security, namely, what we write and what the developers are working on - these are the shortcomings and vulnerabilities of the application itself.
The direction of SDL or SDLC - Security development lifecycle - was developed by Microsoft. The diagram shows the canonical SDLC model, the main task of which is to take part in security at every stage of development, from requirements to release and release to production. Microsoft realized that there are too many bugs in the prom, there are more of them and we need to do something with it, and suggested this approach, which has become canonical.
Application Security and SSDL are not aimed at detecting vulnerabilities, as is commonly believed, but to prevent their occurrence. Over time, the canonical approach from Microsoft was improved, developed, and a deeper, detailed immersion appeared.
Canonical SDLC is highly detailed in various methodologies - OpenSAMM, BSIMM, OWASP. Methodologies are different, but, in general, are similar.
Building Security In Maturity Model
I prefer BSIMM most of all - Building Security In Maturity Model . The basis of the methodology is the division of the Application Security process into 4 domains: Governance, Intelligence, SSDL Touchpoints and Deployment. In each domain there are 12 practices, which are presented in the form of 112 activities.
Each of the 112 activities has 3 levels of maturity : beginner, intermediate and advanced. All 12 practices can be studied by sections, select important things for you, understand how to implement them and gradually add elements, for example, static and dynamic code analysis or code review. You paint the plan and work quietly on it as part of implementing the selected activities.
Why DevSecOps
DevOps is a big process in which you need to take care of security.
DevOps initially assumed security checks. In practice, the number of security teams was much smaller than they are now, and they did not act as participants in the process, but as a control and supervisory body that makes demands on it and checks the quality of the product at the end of the release. This is a classic approach in which security teams were behind the wall from development and did not take part in the process.
The main problem is that the information security is separate from the development. This is usually a kind of IB circuit and there are 2-3 large and expensive tools in it. Every six months, the source code or application arrives that needs to be checked, and pentests are made once a year. This all leads to the fact that the timing of the release in the industry is postponed, and a huge number of vulnerabilities from automated tools fall out on the developer. All this can not be disassembled and repaired, because even in the previous six months the results were not dismantled, and here a new batch.
In the course of our company's work, we see that safety in all areas and industries understands that it is time to catch up and spin with the development in one wheel - in Agile . The DevSecOps paradigm fits nicely on the methodology of agile development, on implementation, support and participation in each release and iteration.
Transition to DevSecOps
The most important word in the Security Development Lifecycle is “process . ” You need to understand this before thinking about buying tools.
Simply incorporating tools into the DevOps process is not enough - interaction and understanding between process participants is important.
More people, not tools
Often planning for a secure development process begins with the selection and purchase of a tool, and ends with attempts to integrate the tool into the current process, which remain attempts. This leads to sad consequences, because all the tools have their own characteristics and limitations.
A common case is when the security department chose a good, expensive tool with ample opportunities, and came to the developers to embed it in the process. But it does not come out - the process is structured so that the limitations of the instrument already purchased do not fall into the current paradigm.
First describe what result you want and how the process will look. This will help to understand the role of the tool and safety in the process.
Start with what is already used.
Before buying expensive tools, look at what you already have. In each company there are safety requirements, which are presented to the development, there are checks, pentests - why not transform all this into a clear and convenient form for everyone?
Usually the requirements - it is a paper talmud, which lies on the shelf. There was a case when we come to the company to see the processes and ask to show the security requirements for the software. The specialist who did this was looking for a long time:
- Now, somewhere in the notes there was a way where this document lies.
As a result, we received the document a week later.
For requirements, checks and other things, create a page, for example, on Confluence - this is convenient for everyone.
Easier to reformat what is already there and use to start.
Use Security Champions
Usually, in an average company, one security officer works for 100-200 developers, who performs several functions and does not physically have time to check everything. Even if he tries his best - he alone will not check all the code that the development generates. For such cases, the concept has been developed - Security Champions .
Security Champions is a person within the development team who is interested in the security of your product.
Security Champion is the entry point to the development team and security evangelist in one person.
Usually, when the security team comes to the development team and indicates an error in the code, he gets a surprised answer:
- And who are you? I see you for the first time. Everything is good with me - I put a “apply” on my senior comrade on code review, we move on!
This is a typical situation, because the elders or just teammates with whom the developer constantly interacts in work and in code review are much more trustworthy. If, instead of the security officer, the Security Champion indicates an error and its consequences, then his word will have more weight.
Also, developers know their code better than any security specialist. For a person who has at least 5 projects in a static analysis tool, it is usually difficult to remember all the nuances. Security Champions know their product: what interacts with what and what to look at first - they are more effective.
So think about embedding Security Champions and expanding the influence of the security team. It is also useful for the champion himself: professional development in the new field, expansion of the technical outlook, improvement of technical, managerial and leadership skills, increase in market value. This is some element of social engineering, your “eyes” in the development team.
Testing stages
A paradigm of 20 on 80 says that 20% of the effort gives 80% of the result. These 20% are application analysis practices that can and should be automated. Examples of such activities are static analysis — SAST , dynamic analysis — DAST, and Open Source control . I'll tell you more about the activity, as well as about the tools, what features we usually encounter when they are introduced into the process, and how to do it correctly.
The main problems of tools
I will highlight issues that are relevant to all tools that need attention. I will analyze them in more detail, so as not to repeat further.
Long time analysis. If it takes 30 minutes to complete the tests and assembly, from the commit to the exit to the prod, the information security checks will take a day. So no one will slow down the process. Consider this feature and draw conclusions.
False Negative High or False Positive. All products are different, all use different frameworks and their own style of writing code. On different code bases and technologies, tools can show different levels of False Negative and False Positive. Therefore, look what exactly in your company and for your applications will show a good and reliable result.
No integration with existing tools . Look at the tools in terms of integrations, so that you already use. For example, if you have Jenkins or TeamCity, check the integration of the tools with this particular software, and not with GitLab CI, which you are not using.
Absence or excessive complexity of customization. If the tool has no API, then why is it needed? Everything that can be done in the interface should be accessible through the API. Ideally, the tool should have the ability to customize the checks.
No roadmap of product development. Development does not stand still, we always use new frameworks and functions, we rewrite the old code into new languages. We want to be sure that the tool we buy will support new frameworks and technologies. Therefore, it is important to know that the product has a real and proper Roadmap development.
Process features
In addition to the features of the tools, consider the features of the development process. For example, interfere with the development - this is a typical error. Let's see what other features should be considered and what the security team should pay attention to.
In order not to disrupt development and release dates, create different rules and different show stoppers — criteria for stopping the build process in the presence of vulnerabilities — for different environments . For example, we understand that the current branch goes to the developer booth or UAT, which means we do not stop and do not say:
- You have vulnerabilities here, you will not go anywhere further!
At this stage, it is important to tell the developers that there are security issues that are worth paying attention to.
The presence of vulnerabilities is not an obstacle for further testing : manual, integration or manual. On the other hand, we need to somehow improve the safety of the product, and that the developers do not forget about what it finds safe. Therefore, sometimes we do this: on the stand, when rolling out to the development environment, we simply notify the development:
- Guys, you have problems, please pay attention to them.
At the UAT stage, we again show warnings about vulnerabilities, and at the stage of exit in the prom we say:
- Guys, we warned several times that you didn’t do anything - we won’t let you go with that.
If we talk about code and dynamics, then it is necessary to show and warn about vulnerabilities only those features and code that was just written in this feature. If the developer moved the button by 3 pixels and we tell him that he has a SQL injection there and therefore it is urgently necessary to correct it - this is wrong. Look only at what is written now, and at the change that comes in the application.
Suppose we have a certain functional defect - the way the application should not work: money is not transferred, when you click on the button there is no transition to the next page or the product is not loaded. Security defects - these are the same defects, but not in the context of the application, and security.
Not all software quality problems are security problems. But all security issues are related to software quality. Sherif Mansour, Expedia.
Since all vulnerabilities are the same defects, they should be located in the same place as all development defects. So forget about reports and scary pdfs that nobody reads.
When I worked in a company that was developing, I received a report from the static analysis tools. I opened it, was terrified, made coffee, looked through 350 pages, closed it and went on to work further. Big reports are dead reports . Usually they do not go anywhere, letters are deleted, forgotten, lost, or the business says that it takes risks.
What to do? Confirmed defects that have been found are simply converted into a form suitable for development, for example, we add them to the backlog in Jira. We prioritize defects and eliminate them in order of priority, along with functional defects and defects of tests.
Static Analysis - SAST
This is an analysis of the code for the presence of vulnerabilities , but it is not the same as SonarQube. We check not only by patterns or style. In the analysis, a number of approaches are used: on the vulnerability tree, on DataFlow , on the analysis of configuration files. This is all that concerns the code itself.
Advantages of the approach : identification of vulnerabilities in the code at an early stage of development , when there are no stands and a finished tool, and the possibility of incremental scanning : scanning a section of code that has changed, and only the feature that we are doing now, which reduces the scan time.
The disadvantages are the lack of support for the required languages.
The necessary integrations that should be in the tools, in my subjective opinion:
- Integration tool: Jenkins, TeamCity and Gitlab CI.
- Development environment: Intellij IDEA, Visual Studio. It is more convenient for the developer not to climb into an incomprehensible interface that still needs to be remembered, but to see all the necessary integrations and vulnerabilities that he has found right in the workplace in his own development environment.
- Code review: SonarQube and manual review.
- Defect trackers: Jira and Bugzilla.
The picture shows some of the best representatives of static analysis.
Not the tools are important, but the process, so there are Open Source solutions that are also good for running the process.
SAST Open Source will not find a huge number of vulnerabilities or complex DataFlow, but you can and should use them when building the process. They help to understand how the process will be built, who will respond to bugs, who will report, who - to report. If you want to conduct the initial step of building the security of your code - use the Open Source solution.
How can this be integrated if you are at the beginning of the journey, you have nothing: neither CI, nor Jenkins, nor TeamCity? Consider integration into the process.
CVS level integration
If you have Bitbucket or GitLab, you can do the integration at the level of Concurrent Versions System .
On event - pull request, commit. You scan the code and, in the build status, show that the security check has passed or failed.
Feedback. Of course, feedback is always needed. If you just performed on the security side, put it in a box and did not tell anyone about it, and then at the end of the month dumped a bunch of bugs - this is not right and not good.
Integration with code review system
Once, we set up a number of important projects as the default reviewer of the technical user AppSec. Depending on whether errors are detected in the new code or there are no errors, at pull request, the reviewer puts the status on “accept” or “need work” - either everything is OK, or you need to refine and refer to what exactly to refine. For integration with the version that goes to the prod, we had a ban on merge if the test on IB is not passed. We included this in the manual code review, and the rest of the process saw the security statuses for this particular process.
Integration with SonarQube
Many have a quality gate for code quality. Here is the same thing - you can do the same gates only for SAST tools. There will be the same interface, the same quality gate, only it will be called the security gate . And also, if you have a process using the SonarQube, you can easily integrate everything there.
CI level integration
Here, too, everything is quite simple:
- At the same level with autotests , unit tests.
- Separation by development stages : dev, test, prod. Different sets of rules can be included, or different fail conditions: we stop the assembly, we do not stop the assembly.
- Synchronous / asynchronous start . We are waiting for the end of testing security tests or not waiting. That is, we simply launched them and move on, and then we receive the status that everything is good or bad.
This is all in a perfect pink world. In real life, this is not, but we strive. The result of performing security checks should be similar to the results of unit tests.
For example, we took a big project and decided that now we will scan it with SAST, OK. We crammed this project into SAST, it gave us 20,000 vulnerabilities, and we decided by good will that everything was fine. 20,000 vulnerabilities are our technical debt. We will put the debt in the box, we will quietly rake and get bugs into the defect-trackers. We will hire a company, do it all ourselves or Security Champions will help us - and technical debt will decrease.
And all newly emerging vulnerabilities in the new code should be eliminated as well as errors in the unit or in the autotests. Relatively speaking, the assembly started, drove out, two tests and two safety tests fell. OK - we went, looked at what happened, corrected one, corrected the second, the next time they drove away - everything is fine, no new vulnerabilities have appeared, the tests are not failed. If this task is deeper and you need to understand it well, or fixing vulnerabilities affects large layers of what lies under the hood: you have introduced a bug in the defect tracker, it is prioritized and corrected. Unfortunately, the world is not perfect and tests sometimes fall.
An example security gate is an analogue of the quality gate, in terms of the presence and number of vulnerabilities in the code.
We integrate with SonarQube - the plugin is installed, everything is very convenient and cool.Integration with development environment
Integration Capabilities:
- Starting a scan from the development environment before commit.
- View results.
- Analysis of the results.
- Synchronization with the server.
Something like this is getting the results from the server.
In our Intellij IDEA development environment, an additional item simply appears, which reports that such vulnerabilities were discovered during the scan. You can immediately edit the code, watch recommendations and Flow Graph . This is all located in the developer’s workplace, which is very convenient - you don’t need to follow the other links and watch something extra.
Open source
This is my favorite topic. Everyone uses the Open Source library - why write a bunch of crutches and bicycles when you can get a ready-made library in which everything is already implemented?
Of course, this is true, but libraries are also written by people, also include certain risks and also there are vulnerabilities, which are periodically or permanently reported. Therefore, there is the next step in Application Security - an analysis of the Open Source component.
Open Source Analysis - OSA
The tool includes three large stages.
Search for vulnerabilities in libraries. For example, the tool knows that we are using some kind of library, and that there are some vulnerabilities in CVE or in bug trackers that relate to this version of the library. If you try to use it, the tool will warn you that the library is vulnerable and advises using a different version where there are no vulnerabilities.
Analysis of license purity. With us this is not very popular yet, but if you work abroad, you can periodically get an atat for using open source components that cannot be used or modified. According to the policy of the licensed library, we cannot do this. Or, if we modified it and use it, then we have to lay out our code. Of course, no one wants to upload the code of their products, but you can protect against this too.
Analysis of the components that are used in the industrial environment. Imagine a hypothetical situation that we finally completed the development and released the last release of our microservice in the prom. He lives there remarkably - a week, a month, a year. We do not collect it, we do not conduct security checks, everything seems to be fine. But suddenly, two weeks after the release, there is a critical vulnerability in the Open Source component, which we use precisely in this assembly, in the industrial environment. If we do not write down what and where we use, then this vulnerability will simply not be seen. Some tools have the ability to monitor vulnerabilities in libraries that are now used in prom. It is very useful.
Opportunities:
- Different policies for different stages of development.
- Component monitoring in industrial environment.
- Control libraries in the organization contour.
- Support for various build systems and languages.
- Analysis of Docker images.
A few examples of area leaders who are engaged in the analysis of Open Source.
The only free one is OWASP's Dependency-Check . You can turn it on at the first stages, see how it works and what it supports. Basically, these are all cloud products, or on-premise, but behind their base they still go to the Internet. They do not send your libraries, but hashes or their values, which they calculate, and fingerprints to their server to receive news of the presence of vulnerabilities.
Process integration
Control perimeter libraries that are downloaded from external sources. We have external and internal repositories. For example, inside Event Central stands Nexus, and we want to have no “critical” or “high” status vulnerabilities inside our repository. You can configure proxying using the Nexus Firewall Lifecycle tool so that such vulnerabilities are cut off and do not fall into the internal repository.
Integration into CI . On the same level as autotests, unit tests and development stages: dev, test, prod. At each stage, you can download any library, use anything, but if there is something hard with a status of "critical", it is possible, you should pay attention to this developers at the stage of prom.
Integration with artifacts : Nexus and JFrog.
Integration into the development environment. The tools you choose should be integrated with development environments. The developer should have access to the scan results from his workplace, or be able to scan himself and check the code for vulnerabilities before committing to CVS.
CD integration. This is a cool feature that I really like and about which I have already said - monitoring the emergence of new vulnerabilities in the industrial environment. It works like this.
We have Public Component Repositories - some tools outside, and our internal repository. We want to have only trusted components. When proxying a request, we check that the library being downloaded does not have vulnerabilities. If it falls under certain policies that we set and necessarily coordinate with the development, then we don’t download it, and the beating comes to use another version. Accordingly, if there is something really critical and bad in the library, then the developer at the installation stage will not receive the library - let him use the version above or below.
- With build, we check that nobody has slipped anything bad, that all components are safe and no one brought anything dangerous on the flash drive.
- In the repository we have only trusted components.
- When deploying, we once again check the package itself: war, jar, DL, or the Docker image that it complies with the policy.
- When entering the industry, we monitor what is happening in the industrial environment: critical vulnerabilities appear or do not appear.
Dynamic Analysis - DAST
The tools of dynamic analysis are fundamentally different from everything that was said before. This is a kind of imitation of user experience with the application. If this is a web application, we send requests, imitating the work of the client, click on the buttons on the front, send artificial data from the form: quotes, brackets, characters in different encoding, to look at how the application works and processes external data.
The same system allows you to check template vulnerabilities in Open Source. Since DAST does not know which Open Source we are using, it simply throws “malicious” patterns and analyzes the server's responses:
- Yeah, there is a problem of deserialization, but not here.
There are big risks in this, because if you run this security test on the same stand with which testers work, unpleasant things can happen.
- High load on the network \ application server.
- No integrations.
- The ability to change the settings of the analyzed application.
- No support for required technology.
- The complexity of the settings.
We had a situation when we finally launched AppScan: we knocked out access to the application for a long time, got 3 accounts and were delighted - finally we’ll check everything! They started the scan, and the first thing that AppScan did was to get into the admin panel, poke through all the buttons, change half of the data, and then kill the server with their own mailform queries. Development with testing said:
- Guys, are you kidding me ?! We gave you accounts, and you put a stand!
Consider the possible risks. Ideally, prepare a separate stand for testing the IB, which will be isolated from the rest of the environment at least somehow, and conditionally check the admin panel preferably in manual mode. This pentest is the remaining percentage of effort that we are not currently considering.
It is worth considering that you can use it as an analogue of load testing. At the first stage, you can turn on the dynamic scanner in 10-15 threads and see what happens, but usually, as practice shows, nothing good.
Several resources that are commonly used.
The Burp Suite is a “Swiss knife” for any security professional. It is used by everyone, and it is very convenient. - enterprise edition. stand alone , - , . , .
: .
mock-, — , .
- — .
- .
- — .
Process
, . — , , OpenSource, - , , Waf .
.
, , , , -.
. , , . , , . , .
API, , , , — AppSec , .
, security- , , , , , . , , — Confluence , Jira -, / CI/CD.
Key Takeaways
. — . , , . — «» , — - high mega super critical, , .
— , . , , . DevSecOps, SecDevOps, .
, : , , , , . — . — .
.
, . , — . - , . , .
— Security Champions .
, , , - — . , . , community, , . , .
.
- False Positive.
- .
- .
- .
- Roadmap .
- .
DevOpsConf 2018. 27 28 DevOpsConf ++ . , , 21 .
Source: https://habr.com/ru/post/448488/
All Articles