Is software security improving?
Yes, some aspects of software security are slowly improving. But, the more accurate answer is: there is still a lot to be done.
What does this yes mean then? That you are spending your efforts in the right way? That applications are getting safer?
Let’s see what kind of things have improved and what’s still missing.
What has been actually improving?
#1 Active usage of secure connections
Applications do rely more and more on secure connections, including internal network communications (which was not the case a couple of years ago). I believe the shift to secure communications has been motivated by different reasons, such as:
- Browser vendors started spreading awareness: not so long ago the major browser vendors started forcing the usage of secure connections. For instance, Google Chrome, Safari and Firefox, are now blocking websites that do not rely on secure connections (you can read more about it here).
- Data breaches and data protection regulations: the number of reported data breaches increased 3 times when compared to 10 years ago and the data leaked in those data breaches is 20 times higher (see Information Is Beautiful for more information) nowadays. Thanks to the GDPR, data breaches have now a huge impact on revenue and business continuity of software companies.
Apart from the growing society awareness and the new legislation, there were also some significant improvements with respect to the configuration of secure connections. Configurations of TLS connections became less complex and cumbersome for people without specific security background. For instance, Mozilla has developed a tool (click here) to automatically derive configuration scripts with recommended cipher suites for certain application servers. Another important advancement was the one created with Let’s Encrypt that allows free generation of public key certificates (by removing the added burden of associated configuration task).
#2 Input/output validation is slowly getting better
Although applications are still far from correctly implementing input validation and output sanitisation, this has somewhat improved over the last years. The widespread usage of ORM frameworks, for instance, is largely contributing to preventing SQL injection attacks.
Modern web applications rely on typed-safe programming languages, which makes the buffer overflows an unlikely issue. Most enterprise system are implemented either in Java and C# (more than 65% of the systems I have analyzed over the last couple of years are written in Java and C#), which, apart from some exceptions, do prevent memory overflows by design.
Moreover, some front-end frameworks (mostly server-side front-end frameworks such as ASP.NET Razor) are doing an amazing job by implementing default input validation. These frameworks implement easy-to-use constructions that validate all the input, and sanitize all the output. Client-side frameworks such as Angular and React also implement (client-side) input validation and output sanitization by default. Although these frameworks do not solve all the input validation issues, they help to create more secure client-side applications.
#3 Identification and session management are improving
Session management has largely improved over the last years, mainly due to the improvement of security frameworks. Also different authentication and authorization mechanisms (such as OpenID, OAuth, SAML, JWT) started to proliferate. Single sign-on (SSO), for instance, is a methodology that largely scales in enterprise environments and highly improves the usability of the applications. Having a centralized mechanism to manage identification and authorization improves organizations mode of work and reduces the risks associated to having users that after a certain time are not allowed to access the applications anymore.
#4 Individuals are more security aware
Individuals in general and the software development community in particular, are becoming more and more security aware. This largely increases the attention on application security. More and more, development teams specify security features in the design phase of the applications. Another common trend is having the OWASP TOP 10 as the baseline for the security requirements. Although these steps do not directly solve all the current application security issues, they do help improving the security of the applications in general.
What is still missing?
#1 Lack of proper logging and audit trail
Sadly, audit trail and logging are rarely important when designing and define the application requirements. Logging is still mostly seen as mechanism to perform debugging and, unfortunately, little or unstructured logging mechanisms are very frequent.
Audit trail becomes then a difficult and sometimes impossible task to perform. Security related events, such as the login and logout events, are hardly logged. On top of that, most of the times, logs are stored in log files that are never actively checked for malicious activities (e.g.: application misusage, access control bypassing, etc.). Integration of the application logs into a centralized logging mechanisms is rarely seen. Despite the existence of integrated solutions, such as ELK (ElasticSearch, LogStash and Kibana) that can be used to perform log management, their integration with the current applications is very scarce.
#2 Decentralized authentication and authorization
Applications implementing centralized authentication and authorization mechanisms are mostly non-existent. Access control checks are done in an unstructured way in the code, sometimes spread over the interface and services layer. This naturally leads to authorization flaws, as developers tend to forget to perform the appropriate checks that prevent elevation of privileges.
Data level access control (i.e., access control depending on the role and type of data that is accessed by the users) is such a cumbersome and time-consuming task, that ends up (almost) never being implemented. The consequence is that, when APIs directly expose database indexes (which in itself is very bad practice), attackers can manipulate these indexes and access unauthorized data (e.g.: retrieve all the database records).
Moreover, user management, which is normally delegated to a third-party application, is most of the times poorly managed. For instance, credentials rely on weak passwords, roles are not managed properly or updated, etc. Password recovery mechanisms are sometimes weak and enable the attackers to easily impersonate valid users.
#3 Protection of application secrets is still weak
Passwords are still being improperly hashed. Weak and custom hashing mechanism are in place and no password salting mechanisms are applied. On top of that, weak crypto algorithms are used. For instance, DES and MD5 considered to be deprecated for many years (because these are very weak crypto algorithms), are still being used. When in place, the implementation of AES (the recommended symmetric encryption standard) is, in 90% of the cases, flawed (e.g.: no random key or IV, same IV used for all encryptions, weak modes of operation, etc.). You can read more about AES implementation issues here.
Additionally, even with the widespread usage of cloud services, which translates into little control over the data modifications, data stored in the databases is never encrypted (or has any sort of integrity protection). This necessary leads to weak protection of sensitive data and consequently to data breaches.
#4 Structural issues in the software development life cycle
Apart from the issues described above, there’re still structural issues in the Secure Software Development Life Cycle (SSDLC).
- Security requirements are poorly defined — the common practice is stating that “the software has to be secure” as requirement or having a security section in the design document being as short as one paragraph with 2–3 lines. The lack of proper specification of the security requirements is perhaps one of the major problems nowadays.
- Lack of threats identification and associated countermeasures — teams still lack the skill of properly identifying the application threats and consequently deriving the appropriated countermeasures. When countermeasures are not specified, they never get to be implemented.
- Peer reviews barely take security into consideration — or sometimes are not even in place at all. The four eyes principle might be time-consuming and lead to a lot of backlog issues. As applications have a reduced time to market, time-pressure and lack of proper knowledge lead to the release of applications that contain many security flaws (that would be easy to detect and fix during the development phase).
- Lack of unit tests for security — the unit testing coverage is in general low. This becomes even lower when considering security features, i.e., no unit testing for security functionalities. Apart from that, integration of Static Analysis Security Tools (SAST) into the CI/CD pipeline is only seen in very mature teams or very critical applications. But even here, teams tend to sometimes ignore the security violations, due to the large amount of false positives these tools output. Filtering out false positives is a tiresome task, seen most of the times as a waste of time.
- Penetration tests or external code reviews are not being done after major releases — sometimes (or most often, to be honest) applications in production for many years, with active development and maintenance are not pentested. External code reviews are something that is even more rare. Another interesting aspect is that good and high quality pentesting and code review companies tend to be expensive, due to the effort required to perform these tasks. So, if very little budget is available, either very low quality analysis are performed (by cheap pentesting services), or not done at all.
So, how to improve further and where to start?
Well, there’s a lot to be done, but in general, trying to improve the Secure Software Development Life Cycle (SSDLC) would largely enhance the security of the applications. Right now, the lack of security knowledge and the lack of proper requirements specification are major issues.
When the SSDLC is flawed from the beginning, the application tends to be vulnerable by design. Investing on improving the development process and on the right people (with the appropriate skills) to guide it, does improve application security.
Plus developers would largely benefit from an increase in the usability of security frameworks. Making the security controls easy to implement and having libraries and frameworks with secure defaults would already be a very good step towards more secure applications.
To summarize, just remember, there is no silver bullet in security. All the aspects and activities of the secure development lifecycle are equally important.
Some motivation about this post
Over the last years, I was in the business of evaluating the security of software systems. This process is and was mostly a manual process of checking the design, source code and development process for the lack of adherence to the security best practices. Reading code every day over the last couple of years (yes, lot’s of manual code reviews), made me gather a significant amount of data around secure coding practices of the enterprise software systems.
In order to understand what is actually changing in terms of application security, I went through the exercise of looking into the data I’ve collected and experience gained. Note that I have mainly analyzed enterprise software applications, with a big focus on web applications.
This exercise helped me to have a better understanding of the most common security practices adopted in industry, as well as, identifying the weak points. My goal is that this analysis can also help improving the security of the applications in general.
Acknowledgements
Many many thanks to my colleagues, Tiago Teles, Jeroen van der Harst and Sander Dorigo for their support and fruitful feedback.
About the author
I’m a crypto consultant in the Crypto Services (CS) team of ABN AMRO. My team is responsible for providing cryptographic services within ABN AMRO and essentially our main goal is to make crypto easy to use.
