Uncomplicate Security for developers using Reference Architectures

Anunay Bhatt
8 min readMar 26, 2021
Figure 1: Security Blueprint for a microservices app on AWS EKS cluster

So you want your developers to build secure applications during SDLC and not as an afterthought after being dinged by Security reviewers. You got this idea of highlighting security controls via reference architectures that can be easily consumed by developers early on in the SDLC life cycle.

But you are concerned! Would this reference architecture be another “lengthy”, “preachy” document with languages like “Thou shall do this, thou shall not do this…” and not really something that can be easily adopted by developers? If you develop a visual reference architecture infographic and give it to your developers, it will still be a “visual” form of a checklist. Not saying that such architectures don’t have a place, they are very helpful for a 10,000 foot view of the security landscape. But that does not really help your developers, does it?

In this blog, we will walk through some of the salient features of a meaningful security reference architecture and the process required to develop one. We will also look at the challenges that one might expect to face while launching a successful security reference architecture program.

Salient features of a successful reference architecture

These are the 7 jewels that must adorn your reference architecture program otherwise there is a high likelihood that your work might not be adopted by developers.

#1 Targeted to the developers — Always think from a developer’s perspective while framing the mission, vision and goals of the reference architecture and also while creating the technical artifacts.

#2 Relatable — Developers are not looking for another security guidance document. Use a demo application, sample code or other relatable means to demonstrate the reference architecture.

#3 Easily adoptable- Automation and scaffolding must be your favorite tools in order to make your reference architecture reachable. Also, reference architecture program must be added at key entry points of SDLC for easy consumption.

#4 Offering benefit to developers — Why would a developer follow your reference architecture? One example of a cherry on top of the cake for developer is — processes are set it place to fast-track reference architecture based apps through the Security review process.

#5 Offering holistic and mappable security- Security controls demonstrated in your reference architecture must be mapped to your company’s GRC standards or in some cases may go above and beyond to create new GRC standards.

#6 Iterative — Multiple deployment models demand multiple “iterative” reference architecture. It’s important to not waste effort on a reference architecture that is not useful to developers. Your process to develop reference architecture must have a minimalistic feasibility analysis phase for Go/No-Go decisions

#7 Measurable — Any measure to demonstrate the adoption of the reference architecture program would work here, but one metric that can convey a good story around showcasing security maturity is — how many apps (Or what percentage of your apps) have baked in security using security reference architecture?

So there you have it — the 7 salient features to make your reference architecture more adoptable to the developers. Next we will see the process of creating a sample reference architecture.

Process for creating reference architecture

Let’s take an example so that it’s easier for you to follow along. Most developers today are using a modular form of application development called microservices. These have also become very popular with the advent of managed Kubernetes clusters offered by public cloud platforms like AWS, GCP and Azure.

So consider a scenario where you are a Security Architect and want your developers to follow your prescribed reference architecture to develop secure microservices apps on a public cloud platform like AWS.

Phase #1 Start with a security blueprint

You can call this anything else but when I say “blueprint”, I am referring to the paper exercise for creating a visual reference architecture. The objective here is to “quickly” develop a good starting point highlighting key security topics but not deep diving into any of the technical areas. I quoted quickly as this phase can help you discuss the efficacy of the reference architecture with minimal effort

For our example, let us see the paper architecture of our ecommerce microservice application:

Figure 2: Application architecture for a demo ecommerce app — Shop sock by Weaveworks

Overlay the paper architecture with your security controls. These controls can be open source, cloud provider specific or specific to your organization. This of course would require good knowledge of the current security tooling of your organization.

Figure 3: Application architecture overlaid with security controls

Quick tip → Before proceeding to next phase, talk to your Security Engineers, Security Architects and other Security Leaders, and discuss the efficacy of the security blueprint. Don’t waste your time and effort on something that is not going to be useful to your developers.

Phase #2 Build a sample application or use an open source one

Developers love using code samples and our objective with reference architecture is just the same to offer them meaningful security integration code samples. But for that we need an app.

For the microservices example, we can select from a number of sample apps that are available online — Sock Shop by Weaveworks, Online Boutique by GCP, etc. The example app architecture we used in the blueprint above is from the Sock Shop demo app by Weaveworks. Shop Sock is a cloud-native microservices demo application. This application is a web-based e-commerce app which allows users to browse different varieties of socks, add them to the cart and buy them.

Quick tip → While onboarding the sample app to your organization, be cautious of using all the developer best practices of your organization like Git setup, CICD tools, third-party package onboarding, golden AMIs, etc. As your sample code would be adopted by developers, you should not employ “shortcuts” for application onboarding

Quick tip → Where possible, reduce operational overhead on your team by using managed services for infrastructure

Phase #3 Integrate security controls with the sample application

This is the phase where you start to secure the sample application and offer guidance to developers while doing so. Outcome of this phase would be “How to” guides illustrated via the sample code of your demo application. Developers can directly copy your code and integrate with their own applications thus making it easier to follow along with these guides. At the same time, the guides must also be mapped to your organization’s GRC standards for backward traceability.

The security choices you make at this phase are very critical and your thought process (pros and cons comparison between tools) must be extensively documented. Some examples are — comparison between container CNIs like Cilium vs Calico, between service mesh like Istio or Linkerd, between managed and unmanaged security tools, etc.

Some example of security controls applicable to our example scenario and also highlighted in Figure 2:

1- Create and use secure CICD pipelines for your application and infrastructure code integrating with secure scanning tools like static analysis, credential scanning, third-party product scanning, secure Git repo, etc.

2- Inject trust certificates into application containers using organization PKI server

3- Integrate front-end with single sign-on server

4- Create least privilege IAM roles in AWS using IAM Access Analyzer

5- Encrypt service-to-service traffic with mTLS using ServiceMesh like Istio

6- Create service to service authorization using ServiceMesh authorization policies

7- Store, retrieve and rotate secrets using Vault

8- Encrypt cloud storage with AWS KMS

9- Monitor service using Zipkin

10- Centralize the storage for infrastructure (Cloudtrail, VPC flow logs, EKS control plane) and OS/app logs

11- Restrict network access with AWS security groups, NACLs, WAF, etc.

12- Secure connectivity to on-premise using AWS DirectConnect

13- Enable fine-grained access control in Kubernetes using Kubernetes access control

14- Detect and remediate cloud resources misconfigurations

Quick tip → Handover the maintenance of security controls documentation to Security engineering teams in your organization like Vault, IAM, PKI, etc. to reduce the operational overhead of such documentation on your team. Also, handover the mapping of compliance solutions to your organization’s GRC team

Phase #4 Make it easy for developers to use your reference architecture and track adoption

Use automation to convert your “How to” guides into one-click templates or create “Hello World” templates which are fully integrated with all the security controls proposed in your reference architecture. You can use any of the following to accomplish this starting with the least resource intensive:

  • Use already existing scaffolding tooling in your company. Partner with the scaffolding team engineers and automate your security controls
  • Offer your controls as infrastructure-as-code scripts (IaaC)
  • Create your own scaffolding templates

Use metrics to track adoption. Some example metrics include — new teams adopting your scaffolded reference architecture templates, increase in documentation views for “How to” guides, increase in Git forks, etc.

Quick tip → Do not recreate the wheel with automation or scaffolding. Use your existing organizational tooling where applicable.

Challenges that you might expect to face

  • Operational and maintenance challenges for sample app—You are a security architecture team, not an application deployment team. Your sample app needs to follow organizational CICD best practices and also needs to maintain patching and other ongoing operational requirements. This would mean extra overhead but it also gives your team a better understanding of developer daily routine and thus increases your team’s developer sympathy.
  • Security controls becoming stale over time — Your team wants to develop new reference architecture but is stuck updating and maintaining the previous one. One possible solution is handing over maintenance to individual security engineering teams.
  • Not enough developers using your reference architectures — It is important to understand developer requirements before jumping into a reference architecture. Talk to developers who have gone through the process and ask for their lessons learned with integrating security. Then make it easy to adopt your reference architecture by tooling or sample code. Refrain from only using theory to educate developers.
  • Handle scope —If your team is not full-stack security then scope your reference architectures accordingly and make it clear the aspects of security that your reference architecture solves for, while caveating the missing areas.

So there you have it, some guidance to Security Architects looking to roll out their own reference architectures for developers. Please feel free to leave comments if you have questions or suggestions to improve this read.

References

Sock Shop by Weaveworks

Online Boutique by GCP

--

--