Overview

This is the second blog post about how to use Generative AI for Securing Cloud Infrastructures, exploring advanced techniques and innovative solutions to fortify cloud security.

Check the first part in How to Use Generative AI for Securing Cloud Infrastructures - Part I.

Let’s continue to deep dive into the rest of the areas where Generative AI secures cloud infrastructure!

4. Vulnerability Assessment with Generative AI

In the realm of vulnerability assessment, Gen AI proves invaluable by simulating potential vulnerabilities within our cloud infrastructure. Through synthetic vulnerability generation, it enables proactive identification and mitigation efforts. Gen AI collaborates with machine learning models like rule-based systems, machine learning classifiers, and deep learning models to enhance vulnerability detection.

Vulnerability Assessment:

Synthetic Vulnerability Generation:

• Description: Gen AI simulates and generates potential vulnerabilities within the cloud infrastructure, aiding proactive identification and mitigation efforts.
• Gen AI’s Role: By generating synthetic vulnerabilities, Gen AI provides insights into potential weaknesses, enabling proactive security measures.

Enhanced Testing and Analysis:

• Description: Gen AI augments vulnerability testing and analysis efforts by leveraging advanced machine learning techniques.
• Gen AI’s Role: Through adaptive learning, Gen AI refines vulnerability detection, ensuring early risk mitigation and comprehensive analysis.

Coverage of Emerging Threats:

• Description: Gen AI models continuously learn from evolving threat intelligence, updating their knowledge of potential vulnerabilities.
• Gen AI’s Role: By staying updated with emerging threats, Gen AI fortifies our defenses, ensuring resilience against the ever-changing threat landscape.

In the domain of threat intelligence, Gen AI analyzes extensive volumes of threat data, identifies patterns, and generates synthetic threat instances, empowering us to uncover hidden vulnerabilities.

4.1. Integration of GenAI in Vulnerability Assessment

In the realm of vulnerability assessment, Generative AI (Gen AI) emerges as a powerful ally, enhancing our understanding and identification of potential weaknesses in the cloud infrastructure:

• Generative AI’s Role:
  • Gen AI employs various techniques, including rule-based systems, machine learning classifiers, and deep learning models, to enhance vulnerability assessment efforts. Rule-based systems utilize predefined rules to detect known vulnerabilities based on patterns or signatures. Machine learning classifiers, trained on labeled vulnerability data, identify patterns and predict the presence of vulnerabilities in new data. Deep learning models analyze complex data such as code snippets or network traffic to identify vulnerabilities or predict vulnerable code segments.
• Utilized Techniques:
  • Rule Based Systems: Gen AI utilizes predefined rules to detect known vulnerabilities based on patterns or signatures, ensuring a proactive approach to vulnerability identification.
  • Machine Learning Classifiers: Gen AI employs machine learning classifiers trained on labeled vulnerability data to identify patterns and predict the presence of vulnerabilities in new data, enabling accurate and efficient vulnerability assessments.
  • Deep Learning Models: Gen AI leverages deep learning models to analyze complex data, including code snippets and network traffic, enabling the identification of vulnerabilities and prediction of vulnerable code segments, enhancing our understanding of potential weaknesses.

Gen AI’s contribution to vulnerability assessment strengthens our cloud defenses, enabling us to fortify our protections against potential exploits and security breaches.

5. Threat Intelligence with Generative AI

Within the realm of threat intelligence, Generative AI can analyze extensive volumes of threat data. Gen AI identifies intricate patterns and generates synthetic threat instances, empowering us to uncover hidden vulnerabilities and anticipate malicious intent.

Pattern Recognition and Analysis:

• Description: Gen AI analyzes large volumes of threat data, identifying patterns and trends for insightful analysis.
• Gen AI’s Role: By recognizing patterns, Gen AI provides valuable insights into evolving threat landscapes, enabling strategic defense mechanisms.

Predictive Analytics:

• Description: Gen AI generates predictions and forecasts based on identified threat patterns, aiding in proactive threat mitigation.
• Gen AI’s Role: Through predictive analytics, Gen AI foretells potential threats, allowing us to prepare and fortify our defenses in advance.

Automated Data Processing:

• Description: Gen AI automates the processing and analysis of vast amounts of threat intelligence data, ensuring efficient utilization.
• Gen AI’s Role: By automating data processing, Gen AI enhances our analytical capabilities, enabling swift responses to emerging threats.

Real-Time Monitoring:

• Description: Gen AI monitors real-time data feeds, detecting anomalies and indicators of compromise for rapid threat response.
• Gen AI’s Role: With real-time monitoring, Gen AI ensures vigilance against immediate threats, allowing us to respond swiftly and decisively.

Contextual Understanding:

• Description: Gen AI captures contextual information and relationships between various threat data elements for nuanced analysis.
• Gen AI’s Role: By understanding context, Gen AI enables precise threat assessments, ensuring accurate responses tailored to specific scenarios.

In summary, armed with insights from vulnerability assessment and knowledge of impending threats from threat intelligence, we can leverage Gen AI’s enhancements to make strategic decisions.

5.1. Integration of GenAI in Vulnerability Assessment

In the realm of vulnerability assessment, Generative AI (Gen AI) emerges as a powerful ally, revolutionizing our ability to identify and mitigate potential weaknesses within our digital fortifications.

• Gen AI’s Role:
  • Gen AI employs advanced Natural Language Processing (NLP) techniques to analyze security reports, blogs, and forums, identifying patterns and trends in threat data. This deep contextual understanding enhances our ability to recognize potential risks and vulnerabilities.
• Utilized Techniques:
  • NLP-based Pattern Recognition and Analysis: Gen AI utilizes NLP algorithms to analyze unstructured threat intelligence data, extracting valuable insights from security reports, blogs, and forums. This analysis helps us identify intricate patterns and anticipate potential vulnerabilities.
  • Cluster Algorithms: Gen AI employs cluster algorithms, such as K-means or hierarchical clustering, to group similar threat intelligence data together based on common attributes. This clustering enables a more organized and nuanced understanding of potential vulnerabilities, allowing for targeted mitigation efforts.
  • Generative Models: Gen AI leverages generative models, including variational autoencoders and generative adversarial networks, to create synthetic threat instances. By generating these instances, Gen AI aids in comprehensive threat modeling, enabling us to anticipate and address potential vulnerabilities effectively.

Gen AI’s integration with NLP, cluster algorithms, and generative models enhances our vulnerability assessment capabilities, ensuring a proactive and robust defense against emerging threats.

6. Security Incident Response with Generative AI

Security incident response involves detecting, investigating, and mitigating security incidents within a cloud infrastructure. Generative AI, with its adaptive learning mechanisms, plays a pivotal role in this domain.

Anomaly Detection:

• Description: Generative AI models learn normal patterns within the cloud infrastructure, identifying anomalies that may indicate potential security incidents.
• Gen AI’s Role: By recognizing deviations, Gen AI aids in early incident detection, enabling prompt response to potential threats.

Real-Time Monitoring:

• Description: Gen AI models continuously monitor network traffic, system logs, and user activities in real time.
• Gen AI’s Role: Its vigilant real-time monitoring allows immediate response to any deviations from established norms, ensuring swift threat mitigation.

Automated Alert Generation:

• Description: Gen AI automatically generates alerts upon detecting anomalies or potential security incidents, ensuring rapid awareness and response.
• Gen AI’s Role: Through automated alert generation, Gen AI enhances situational awareness, enabling quick and effective incident response.

Incident Triage and Prioritization:

• Description: Gen AI assists in triaging and prioritizing security incidents based on severity, impact, or potential risks.
• Gen AI’s Role: By prioritizing incidents, Gen AI guides efficient response strategies, allowing us to focus resources where they are most needed.

Root Cause Analysis:

• Description: Gen AI analyzes patterns and anomalies associated with security incidents, facilitating in-depth understanding and effective resolution.
• Gen AI’s Role: Through root cause analysis, Gen AI uncovers the underlying causes of incidents, enabling targeted response and preventing recurrence.

Threat Hunting:

• Description: Gen AI analyzes historical incident data and generates synthetic instances of potential threats, empowering proactive threat hunting initiatives.
• Gen AI’s Role: By simulating threats, Gen AI aids in proactive hunting, allowing us to anticipate and mitigate emerging threats before they manifest.

Decision Support:

• Description: Gen AI offers insights and recommendations based on learned patterns and historical incident data, guiding informed decision-making during incident response.
• Gen AI’s Role: By providing decision support, Gen AI assists security professionals in making informed choices, optimizing incident response strategies.

Continuous Learning and Adaptation:

• Description: Gen AI continuously learns and adapts to evolving threats and attack techniques, ensuring up-to-date and effective incident response strategies.
• Gen AI’s Role: Through continuous learning, Gen AI stays ahead of emerging threats, allowing us to adapt and respond effectively to the ever-changing threat landscape.

6.1. Integration of GenAI in Security Incident Response

In the realm of security incident response, Generative AI (Gen AI) emerges as a powerful ally, enhancing our capabilities to combat cyber threats and safeguard our digital domain.

• Gen AI’s Role:
  • Gen AI learns from historical incident data and generates synthetic instances, enabling incident simulation, response planning, and decision-making.
• Utilized Techniques:
  • Rule-Based Systems: Gen AI utilizes predefined rules and conditions to detect and respond to known security incidents, ensuring rapid response based on established protocols.
  • Machine Learning Classifiers: Gen AI employs machine learning classifiers trained on labeled incident data, identifying patterns and predicting the likelihood and severity of security incidents.
  • Natural Language Processing (NLP) Models: Gen AI analyzes unstructured incident reports, security logs, and threat intelligence data, extracting valuable insights using techniques like named entity recognition and sentiment analysis.
  • Deep Learning Models: Gen AI utilizes deep learning models for image or text-based analysis of incident data, enhancing our understanding of complex incidents and their underlying patterns.
  • Graph Analytics: Gen AI employs graph-based models, representing incident data as interconnected nodes and edges, facilitating the analysis of relationships, dependencies, and potential attack paths.
  • Reinforcement Learning: Gen AI models, using reinforcement learning techniques, learn optimal response actions through interactions with simulated incident response environments, guiding automated decision-making during incidents.
  • Generative AI Models: Gen AI incorporates generative models like autoencoders, variational autoencoders, and generative adversarial networks, enabling the generation of synthetic instances for in-depth incident analysis.

Gen AI’s profound understanding of incidents, coupled with its adaptive nature, strengthens our incident response capabilities, enabling us to thwart even the most sophisticated cyber adversaries and maintain the security of our digital realm effectively.

7. Access Control with Generative AI

Through its advanced techniques and adaptive learning, Gen AI empowers us to secure our cloud infrastructure against unauthorized access and data breaches.

Anomaly Detection:

• Description: Gen AI models learn normal patterns within the cloud infrastructure, identifying anomalies that may indicate potential security incidents.
• Gen AI’s Role: By recognizing deviations, Gen AI aids in early incident detection, enabling prompt response to potential threats.

Real-Time Monitoring:

• Description: Gen AI models continuously monitor network traffic, system logs, and user activities in real time.
• Gen AI’s Role: Its vigilant real-time monitoring allows immediate response to any deviations from established norms, ensuring swift threat mitigation.

Adaptive Access Policies:

• Description: Gen AI adapts access control policies based on observed user behaviors and context.
• Gen AI’s Role: Its adaptive policies ensure dynamic and context-aware access decisions, enhancing the security of our cloud resources.

7.1. Integration of GenAI in Access Control

In the realm of access control, Generative AI (Gen AI) stands as a stalwart guardian, fortifying our defenses against unauthorized access attempts and ensuring the integrity of our digital kingdom.

• Gen AI’s Role:
  • Gen AI plays a pivotal role in access control, ensuring the security of our digital domain through:
    • Learning from user attributes, access patterns, and resource properties to dynamically adapt access control policies.
    • Detecting anomalous behaviors and making personalized access decisions, enhancing our ability to thwart unauthorized access attempts.
• Utilized Techniques:
  • Rule-Based Systems: Employing predefined rules and conditions, Gen AI ensures swift response to known access patterns, controlling access based on specific attributes and patterns.
  • Machine Learning Classifiers: Gen AI utilizes machine learning classifiers trained on labeled access data to discern intricate patterns, facilitating precise and adaptive access control decisions.
  • Neural Networks: Leveraging neural networks, Gen AI comprehends complex patterns, user behaviors, and contextual information, enabling nuanced access control decisions rooted in learned representations.
  • Reinforcement Learning: Gen AI harnesses reinforcement learning models, dynamically adjusting access control rules and policies based on feedback received during training. This adaptive approach optimizes access decisions, enhancing the flexibility of our access control mechanisms.

Gen AI’s mastery of rule-based systems, machine learning classifiers, neural networks, and reinforcement learning fortifies our access control mechanisms, ensuring precise and efficient management of access privileges within our digital realm.

8. Data Protection with Generative AI

In the realm of data protection, Generative AI (Gen AI) serves as a formidable ally, safeguarding our sensitive information from unauthorized access, use, or disclosure.

Anomaly Detection and Data Usage:

• Description: Gen AI models learn the normal patterns of data access and usage within our cloud infrastructure, identifying anomalous behaviors indicative of unauthorized data access or data leakage attempts.
• Gen AI’s Role: By detecting deviations, Gen AI enhances our ability to promptly identify and respond to potential data breaches, ensuring the confidentiality of our sensitive data.

Privacy Preserving Data Sharing:

• Description: Gen AI techniques such as secure multi-party computation or federated learning enable collaborative analysis and decision-making on sensitive data while preserving privacy.
• Gen AI’s Role: These techniques facilitate secure data sharing and analysis, allowing for meaningful insights without compromising the privacy of individual data records.

Synthetic Data Generation:

• Description: Gen AI models generate synthetic data that retains the statistical properties and patterns of the original data, enabling testing, development, or sharing without exposing actual sensitive information.
• Gen AI’s Role: By providing realistic yet synthetic data, Gen AI minimizes the need to expose actual sensitive data, reducing the risk of data breaches while supporting various applications and analyses.

Data Loss Prevention:

• Description: Gen AI models aid in detecting potential data loss incidents, such as unauthorized data transfers, abnormal data deletion, or unusual data access patterns.
• Gen AI’s Role: By identifying suspicious data-related activities, Gen AI enhances our capability to prevent data loss, ensuring the integrity and confidentiality of our critical information.

8.1. Integration of GenAI in Data Protection

In the realm of data protection, Generative AI (Gen AI) proves invaluable, reinforcing our defenses against potential data breaches and ensuring the secure handling of our digital assets.

• Gen AI’s Role:
  • Gen AI plays a crucial role in data protection, utilizing techniques like:
    • Encryption Algorithms: Gen AI employs cryptographic algorithms to transform data into unreadable formats, ensuring secure data transmission and storage.
    • Anonymization Models: These models enable secure sharing and analysis of data while preserving individual privacy, safeguarding sensitive information from unauthorized access.
    • Differential Privacy Models: Gen AI introduces noise to query responses, preventing the identification of individual data records and preserving data privacy.
    • Generative AI Models: Gen AI generates synthetic data, retaining statistical properties of the original data, enabling safe testing, development, and sharing without exposing actual sensitive information.

8.1. Integration of GenAI in Data Protection

In the domain of data protection, Generative AI (Gen AI) plays a pivotal role as a robust safeguard, reinforcing our endeavors to secure sensitive information, prevent unauthorized access, and uphold the confidentiality and integrity of our digital assets.

• Gen AI’s Role:
  • Gen AI serves a vital role in data protection through the Synthetic Data Generation. Gen AI generates synthetic instances of sensitive data, preserving statistical properties and patterns, enabling secure testing, development, and sharing without exposing actual sensitive information.
• Utilized Techniques:
  • Encryption Algorithms: Gen AI employs robust cryptographic algorithms, transforming data into unreadable formats, guaranteeing secure data transmission and storage, preventing unauthorized access.
  • Anonymization Models: These models enable confidential data sharing and analysis while preserving individual privacy, ensuring secure collaborative decision-making without compromising sensitive information.
  • Differential Privacy Models: Gen AI introduces noise to query responses, safeguarding individual data records and ensuring data privacy, making it challenging to identify specific data points.

And with that, we conclude our second blog post on “How to Use Generative AI for Securing Cloud Infrastructures.”

NOTE: Opinions expressed in this blog are my own and do not necessarily reflect that of the company I work for.

Happy AI/MLing!

1. What is Generative AI

Generative Artificial Intelligence (Gen AI) refers to a branch of artificial intelligence focused on generating new and original data based on patterns and examples observed in existing data sets.

Gen AI models use advanced algorithms to learn these patterns and then create synthetic instances of data that resemble the original dataset. This technology is particularly valuable for tasks such as generating realistic images, simulating human-like speech, and enhancing predictive analytics.

In the context of cloud security, Gen AI models can be employed to create simulated threats, predict user behavior, and identify potential vulnerabilities, enhancing overall threat detection and response capabilities within cloud infrastructures.

2. Areas where Generative AI Secures Cloud Infrastructure

In our exploration of securing cloud infrastructure with Generative Artificial Intelligence, we delve into five crucial domains:

2.1. Threat Detection:

• Overview: Threat detection involves proactive identification and response to security threats. Generative AI, particularly generative adversary networks, aids by crafting synthetic instances of threats, enabling early risk mitigation.
• Insights: By analyzing patterns and generating synthetic threat instances, Generative AI enhances the ability to predict, prevent, and respond to diverse cyber threats effectively.

2.2. User Behavior Analysis:

• Overview: User Behavior Analysis focuses on understanding user interactions within the cloud environment. Generative AI, like variational autoencoders, helps decipher user behavior patterns by capturing essential data aspects.
• Insights: Generative AI assists in distinguishing normal behavior from anomalies, enabling the identification of insider threats and enhancing overall security awareness. It predicts user intentions, contributing to a proactive security posture.

2.3. Vulnerability Assessment:

• Overview: Vulnerability assessment evaluates system weaknesses. Generative AI simulates potential attacks, aiding in the identification and prioritization of vulnerabilities for robust security measures.
• Insights: By generating synthetic instances of vulnerabilities, Generative AI assists in comprehensive vulnerability assessment. It allows cloud engineers to prioritize remediation efforts effectively, strengthening the defenses.

2.4. Threat Intelligence:

• Overview: Threat intelligence involves gathering insights to understand cyber threats. Generative AI processes vast data, extracting actionable intelligence, enabling security teams to anticipate and counteract evolving threats.
• Insights: Generative AI sifts through data, identifying meaningful patterns to enhance threat intelligence. By generating synthetic threat scenarios, it aids in proactive measures, ensuring vigilance against emerging threats.

2.5. Security Incident Response:

• Overview: Security incident response focuses on rapid detection, analysis, and mitigation of security incidents. Generative AI helps prepare security teams by simulating realistic incident scenarios, enhancing incident response preparedness.
• Insights: Generative AI models create lifelike incident scenarios, allowing security professionals to practice response strategies. It ensures that response teams are well-equipped to handle real-world security incidents swiftly and efficiently.

In this strategic approach, Generative AI seamlessly integrates into the cloud security landscape, offering invaluable insights and predictive capabilities. Its contribution, when harmonized with human expertise, fortifies the security measures, ensuring resilience in the face of evolving cyber threats.

3. Threat Detection with Generative AI

Within the realm of Threat Detection, Generative AI (Gen AI) plays a pivotal role in deciphering intricate patterns and anomalies in user behavior within the cloud environment.

By employing Gen AI, cloud security professionals gain profound insights into user interactions, elevating their analyses to a comprehensive level. Gen AI’s adaptive learning mechanisms enable the detection of subtle deviations and suspicious activities, ensuring early threat detection.

Its ability to distinguish between normal behaviors and potential threats enhances the precision of threat detection systems. Incorporating Gen AI in Threat Detection not only fortifies cloud security but also empowers security teams to proactively mitigate emerging cyber threats.

Anomaly Detection:

• Description: Anomaly detection identifies deviations or anomalies from normal patterns in cloud infrastructure behavior, indicating potential threats.
• Gen AI’s Role: Gen AI models are trained on normal cloud system behaviors. By monitoring generated output against real-time data, any deviations can be detected, pointing towards potential threats.

Intrusion Detection:

• Description: Intrusion detection focuses on identifying unauthorized access attempts and suspicious activities in network traffic patterns.
• Gen AI’s Role: Generative AI models analyze network traffic patterns and learn from known attack patterns. By recognizing and flagging similar patterns in real-time traffic, Gen AI helps detect intrusions effectively.

Malware Detection:

• Description: Malware detection targets identifying malicious software instances that aim to disrupt and damage data and infrastructure.
• Gen AI’s Role: Gen AI models are trained on malware samples or behaviors to generate synthetic malware instances. By comparing real-time data with these synthetic instances, Gen AI helps in identifying and responding to potential malware threats.

3.1. Integration of GenAI in Holistic Threat Detection

In the domain of holistic threat detection, Generative AI (Gen AI) stands as a cornerstone, bolstering cloud security through advanced techniques and imaginative solutions:

• Generative AI’s Role:
  • Gen AI’s adaptive learning refines threat detection by generating simulated threats based on learned patterns and anomalies, ensuring early risk mitigation.
• Utilized Techniques:
  • Autoencoders: Uncover latent patterns in data, enhancing anomaly detection and providing valuable insights for proactive security measures.
  • Variational Autoencoders: Capture intricate data distributions, enabling the generation of synthetic threat instances for comprehensive threat modeling.
  • Generative Adversary Networks: Craft vivid replicas of potential threats, aiding in understanding adversary tactics and strengthening incident response strategies.

Gen AI’s synergy with autoencoders, variational autoencoders, and generative adversary networks fortifies the cloud infrastructure, ensuring a resilient defense against malicious adversaries.

4. User Behavior Analysis with Generative AI

In the domain of User Behavior Analysis, we gain insight into understanding and interpreting user interactions within the cloud environment, employing Generative AI (Gen AI) to elevate these analyses to a comprehensive level:

Capturing Intricate User Behavior:

• Description: User Behavior Analysis focuses on studying how users engage with the cloud system, identifying both normal patterns and deviations that might indicate security risks.
• Gen AI’s Role: Gen AI models meticulously learn from vast datasets of user interactions, capturing intricate behavioral nuances. By discerning patterns from this data, Gen AI helps in understanding complex user behavior within the cloud infrastructure.

Anomaly Detection and Identification:

• Description: Anomaly detection involves spotting irregular user actions that deviate from established norms, potentially signifying security threats.
• Gen AI’s Role: Gen AI models analyze user behavior patterns, distinguishing between normal activities and anomalies. By recognizing deviations, Gen AI aids in the rapid identification of suspicious actions, which is crucial for proactive threat mitigation.

Adaptability to User-Specific Patterns:

• Description: User-specific behavior patterns are unique and can evolve over time. Understanding these individual patterns is essential for effective security analysis.
• Gen AI’s Role: Gen AI adapts to user-specific behaviors by continuously learning from individual interactions. This adaptability ensures that security measures remain tailored to each user, enhancing the overall accuracy of threat detection.

Contextual Understanding of User Actions:

• Description: Context plays a vital role in understanding user behavior. Analyzing user actions in specific contexts, such as resource access or network interactions, provides valuable insights.
• Gen AI’s Role: Gen AI models capture the context and dependencies of user behavior, enabling precise analysis of actions related to cloud resources, access privileges, and network interactions. This contextual understanding enhances the accuracy of detecting abnormal user behavior.

Early Detection of Insider Threats:

• Description: Insider threats occur when authorized users engage in malicious activities. Detecting these threats early is crucial for preventing data breaches and other security incidents.
• Gen AI’s Role: Gen AI excels in early detection by identifying subtle changes in user behavior, signaling potential insider threats. Its ability to predict user intentions aids in the proactive identification of security risks originating from within the organization.

Continuous Learning and Adaptation:

• Description: User behavior can change over time due to various factors. Continuous adaptation to these changes ensures that security measures remain effective.
• Gen AI’s Role: Gen AI models continually learn from new user behavior data, updating their understanding of normal behavior and adapting to changes. This continuous learning ensures that the cloud infrastructure’s security remains resilient against evolving user-based threats.

4.1. Integration of GenAI in the User Behavior Analysis domain

In the domain of User Behavior Analysis, the integration of Generative AI (Gen AI) and advanced techniques enhances the understanding of user interactions, fortifying cloud security:

• Generative AI Integration:
  • Gen AI’s adaptive learning refines user behavior analysis by processing vast datasets, ensuring nuanced insights into cloud interactions.
• Utilized Techniques:
  • Hidden Markov Models: Unravel complex sequential patterns in user actions, aiding in behavior prediction.
  • Recurrent Neural Networks (RNNs): Capture intricate dependencies in user behavior sequences, enhancing predictive accuracy.
  • Long Short-Term Memory Networks (LSTMs): Effectively handle long-term patterns, ensuring comprehensive analysis of user interactions.
  • Self-Organized Maps (SOMs): Facilitate clustering and visualization of high-dimensional user data, enabling in-depth understanding.

This integration empowers the cloud infrastructure with enhanced security measures, safeguarding against potential threats and reinforcing the cloud defenses.

And with that, we conclude our first blog post on “How to Use Generative AI for Securing Cloud Infrastructures.”

In this blog post, we delved into the profound influence of Generative AI on cloud security. Our next blog post will unravel the mysteries of Vulnerability Assessment, delve into the depths of Threat Intelligence, and prepare us for the challenges of Security Incident Response.

NOTE: Opinions expressed in this blog are my own and do not necessarily reflect that of the company I work for.

Happy AI/MLing!

1. Overview

In this blog post, the primary focus is on creating a sophisticated ChatBot application with seamless integration into Azure OpenAI. The goal is to develop, build, and deploy a ChatBot that serves as a user-friendly FrontEnd, powered by Gradio, a Python library known for simplifying the creation and sharing of applications.

A crucial component enhancing this ChatBot’s capabilities is LangChain, a versatile framework tailored for building applications driven by language models. LangChain empowers applications to establish contextual awareness by connecting language models to various sources of context, such as prompt instructions, few-shot examples, and relevant content. This contextual understanding ensures the ChatBot’s responses are grounded effectively, enhancing user interaction.

The unique aspect of this ChatBot lies in its backend, where a robust GPT Model is deployed on Azure OpenAI. This integration ensures a smooth user experience, leveraging the capabilities of OpenAI’s cutting-edge technology within Azure’s reliable environment.

This integration highlights the power of Azure Red Hat OpenShift, which serves as the platform for deploying this ChatBot application. By harnessing the potential of Large Language Models like GPT, this blog demonstrates the innovative possibilities that arise when advanced AI technology meets the secure infrastructure provided by Azure OpenAI and Azure Red Hat OpenShift.

Throughout the blog, readers will find detailed steps on how to create and deploy such an application, making it a comprehensive guide for developers and enthusiasts eager to explore the synergy between Azure OpenAI and Azure Red Hat OpenShift.

The blog aims to inspire and empower readers to harness the full potential of AI-driven applications while ensuring a seamless integration process, from development to deployment, in the Azure ecosystem.

2. ARO AI ChatBot Azure OpenAI Components

2.1 Azure OpenAI Overview

Azure OpenAI Service offers convenient REST API access to OpenAI’s advanced language models, such as GPT-4, GPT-3.5-Turbo, and Embeddings series. The GPT-4 and GPT-3.5-Turbo models are now widely available. These models can be tailored for various tasks like content creation, summarization, semantic search, and translating natural language to code. Users can utilize the service via REST APIs, Python SDK, or the web-based interface in Azure OpenAI Studio.

Azure Red Hat OpenShift delivers on-demand, fully managed OpenShift clusters that are highly available, with joint monitoring and operation by Microsoft and Red Hat. At its core, it utilizes Kubernetes. OpenShift enhances Kubernetes by adding valuable features, transforming it into a turnkey container platform as a service (PaaS) that greatly enhances the experiences of both developers and operators.

2.1.1 Comparing Azure OpenAI vs OpenAI

Azure OpenAI Service provides customers access to sophisticated language AI models like OpenAI GPT-4, GPT-3, Codex, DALL-E, and Whisper, all within the secure and reliable environment of Azure. In collaboration with OpenAI, Azure OpenAI co-develops APIs, ensuring seamless compatibility and transition between models.

With Azure OpenAI, customers benefit from the robust security features of Microsoft Azure while utilizing identical models as OpenAI. Azure OpenAI offers private networking, availability in specific regions, and responsible AI content filtering, enhancing the overall user experience and ensuring responsible usage of AI technology.

2.2 Gradio

A highly effective approach for showcasing your machine learning model, API, or data science workflow to others involves developing an interactive application that enables users or peers to experiment with the demo directly through their web browsers.

Gradio, a Python library, offers a streamlined solution for constructing such demos and facilitating easy sharing. In many cases, achieving this only requires a concise snippet of code.

If you want to take a look at other Gradio Apps, check my blog post about Deploying and Testing Machine Learning Applications in Kubernetes with Gradio!

2.3 LangChain

LangChain stands as a versatile framework designed for developing applications driven by language models. Its core functionalities enable applications to:

• Contextual Awareness: LangChain facilitates the connection of language models to diverse sources of context, including prompt instructions, few-shot examples, and relevant content. This enables applications to respond in a manner grounded in the provided context, enhancing their overall effectiveness.

• Intelligent Reasoning: By leveraging LangChain, applications gain the ability to rely on language models for reasoning. This involves determining appropriate responses and actions based on the provided context, enhancing the application’s decision-making process significantly.

The key value propositions offered by LangChain include:

• Modular Components: LangChain provides abstract structures for interacting with language models, coupled with a variety of implementations for each structure. These components are modular and user-friendly, ensuring ease of use, whether integrated within the LangChain framework or utilized independently.

• Pre-configured Chains: LangChain offers pre-built chains, which are structured combinations of components designed to accomplish specific high-level tasks. These ready-to-use chains simplify the application development process, allowing developers to focus on specific functionalities without the hassle of building complex architectures from scratch.

In essence, LangChain streamlines the development process, offering a flexible and efficient approach to creating context-aware and intelligent applications powered by language models.

2.4 Azure Red Hat OpenShift

The Microsoft Azure Red Hat OpenShift service enables you to deploy fully managed OpenShift clusters.

Azure Red Hat OpenShift is jointly engineered, operated, and supported by Red Hat and Microsoft to provide an integrated support experience. There are no virtual machines to operate, and no patching is required. Master, infrastructure, and application nodes are patched, updated, and monitored on your behalf by Red Hat and Microsoft. Your Azure Red Hat OpenShift clusters are deployed into your Azure subscription and are included on your Azure bill.

When you deploy Azure Red Hat OpenShift 4, the entire cluster is contained within a virtual network. Within this virtual network, your master nodes and worker nodes each live in their own subnet. Each subnet uses an internal load balancer and a public load balancer.

3. ARO AI ChatBot with Azure OpenAI: Demo Analysis

Let’s shift our focus from theory to practical application and explore how this ChatBot operates and interacts in real-time.

Once deployed within our Azure Red Hat OpenShift cluster, the ARO AI ChatBot application becomes accessible through a specific URL. This application, powered by Azure OpenAI as its backend, offers a user-friendly interface for interaction.

In the provided images, we witness the ChatBot in action. In the first screenshot, we posed a straightforward yet intriguing question: “What is Azure Red Hat OpenShift?”.

The Gradio App, functioning as the frontend, utilizes LangChain libraries internally to connect with our deployed GPT 3.5 model in Azure OpenAI:

Specifically, we deployed the gpt-35-turbo model, utilizing the 0301 version, hosted in the Azure France Central region.

But how can we be certain that our ChatBot is effectively utilizing the Azure OpenAI GPT Model as its backend?

To validate this, we can delve into the ARO Console and inspect the ChatBot’s logs:

Upon careful observation, the logs reveal the ChatBot’s process: it sends the Human Message using LangChain libraries to the Azure OpenAI URL. The GPT Model in Azure OpenAI generates the response, which is then relayed back to the “human,” completing the ChatBot’s interaction loop.

This seamless integration showcases the synergy between LangChain libraries, Gradio frontend, and Azure OpenAI backend, enabling a dynamic and interactive user experience.

In our upcoming blog post, we will delve deeply into the integration components of the demonstration and provide comprehensive insights into how you can deploy your very own version in your Azure Red Hat OpenShift cluster!

Stay tuned for an in-depth exploration of the deployment process and harness the power of this integration for your projects.

NOTE: Opinions expressed in this blog are my own and do not necessarily reflect that of the company I work for.

Happy AI/ML-ing!

Overview

In the dynamic world of hybrid multi-cloud environments, organizations are constantly seeking robust solutions to expose their customer applications securely. In this blog post, we will focus on the critical topic of exposing customer applications within Private Azure Red Hat OpenShift (ARO) clusters and shed light on the pivotal role played by Application Gateway in enabling this process.

Application Gateway, a versatile component of the Azure ecosystem, serves as a powerful tool for secure application exposure. It seamlessly integrates with Private ARO clusters, aligning with the overarching connectivity strategy of Open Hybrid Cloud.

By leveraging Application Gateway, organizations can unlock a lot of benefits when it comes to exposing customer applications with Private ARO Cluster. Firstly, it acts as a highly efficient load balancer, intelligently distributing incoming traffic across multiple instances of applications running within the ARO cluster. This ensures optimal performance and availability, even during high demand scenarios.

Moreover, Application Gateway provides robust security features, safeguarding customer applications from external threats. It offers comprehensive SSL/TLS termination, enabling end-to-end encryption for enhanced data protection. Additionally, its Web Application Firewall (WAF) functionality protects against common web vulnerabilities, providing an additional layer of defense.

The objective of this blog post is to demonstrate exposing some Customer Applications deployed in a Private ARO cluster, where the requirement is to expose only the Application itself, not the ARO API Kubernetes Ingress or any other *.apps routes.

Also, the certificates need to be taken into consideration, since we will NOT use a Custom Domain for our ARO Cluster. We will be using a Let’s Encrypt certificate with the APP FQDN, and we will place the certificate in the AppGW and in the OpenShift App Route in the ARO cluster.

Azure Application Gateway

Azure Application Gateway is a load balancer designed for managing web traffic directed towards your web applications. Unlike traditional load balancers that operate at the transport layer (OSI layer 4 - TCP and UDP) and direct traffic solely based on source IP address and port to a destination IP address and port, Application Gateway goes a step further.

Application Gateway has the capability to make routing decisions based on additional attributes of an HTTP request, such as the URI path or host headers.

Prerequisites

• ARO Private Cluster (use 10.0.10.0/16 as VNet CIDR)
• Jumphost VM with Public IP

Setting Environment Variables

• Set some specific environment variables for the ARO environment:

export NAMESPACE=aro-app-agw
export AZR_CLUSTER=aro-$USER
export AZR_RESOURCE_LOCATION=eastus
export AZR_RESOURCE_GROUP=aro-$USER-rg
export AppGW_CIDR="10.0.10.0/23"
export AppGW_SUBNET="Ingress-subnet"
export ARO_VNET_NAME="aro-$USER-vnet"
export APP_NAME="aro-hello-openshift"
export DNS_ZONE_NAME="test.openshiftdemo.dev"
export APPGW_DOMAIN="$APP_NAME.$DNS_ZONE_NAME"
export AppGW_PIP="AppGW-pip"
export AZR_DNS_RESOURCE_GROUP="mobb-dns"
export EMAIL=username.taken@gmail.com

NOTE: Customize these variables for your own deployment!

AppGW Networking and Private DNS Zones

• Create Subnet for AppGW:

az network vnet subnet create \
  --resource-group $AZR_RESOURCE_GROUP \
  --vnet-name $ARO_VNET_NAME \
  --name $AppGW_SUBNET \
  --address-prefixes $AppGW_CIDR \
  --service-endpoints Microsoft.ContainerRegistry

NOTE: Since the AppGW and ARO subnets share the same VNet, there is no need to add a peering. If you want to split them between two different VNets instead of subnets, please remember to add the VNet peering between them.

• Create a static public IP address for the Application Gateway LB:

az network public-ip create \
  --resource-group $AZR_RESOURCE_GROUP \
  --name $AppGW_PIP \
  --allocation-method Static \
  --sku Standard

• Create a Private DNS Zone with the same public DNS_Zone_Name as we will be using in the blog post:

az network private-dns zone create \
    --resource-group $AZR_RESOURCE_GROUP \
    --name $DNS_ZONE_NAME

This is needed because the AppGW needs to reach the internal ARO LB when the Backend Rule is applied in the AppGW for our Custom Domain Application.

• Retrieve the ARO Ingress Internal IP Load Balancer:

INGRESS_IP="$(az aro show -n $AZR_CLUSTER -g $AZR_RESOURCE_GROUP --query 'ingressProfiles[0].ip' -o tsv)"
echo $INGRESS_IP

• Add a record for our FQDN to a private DNS zone pointing to the ARO Ingress Internal IP LB:

az network private-dns record-set a add-record \
  --resource-group $AZR_RESOURCE_GROUP \
  --zone-name $DNS_ZONE_NAME \
  --record-set-name "$APP_NAME" \
  --ipv4-address $INGRESS_IP

NOTE: We are using the same $APP_NAME (in our case aro-hello-openshift) with the same private DNS zone (test.openshiftdemo.dev), pointing to the Azure Internal LB that will load balance to the Workers where the ARO OpenShift Routers are (OpenShift Ingress Controllers that manage the HAProxy OpenShift Routers).

• Link Private DNS Zone to ARO Virtual Network:

az network private-dns link vnet create \
    --resource-group $AZR_RESOURCE_GROUP \
    --zone-name $DNS_ZONE_NAME \
    --name private-dnszone-link-$ARO_VNET_NAME \
    --virtual-network $ARO_VNET_NAME \
    --registration-enabled false

Application GW and WAF policy

Now that we have our Networking and the DNS resolution deployed and configured, let’s create the Application Gateway LB and the WAF Policies.

Create the Application Gateway Load Balancer and WAF policies

• Create a Web Application Firewall (WAF) policy for the Application Gateway:

az network application-gateway waf-policy create \
  --resource-group $AZR_RESOURCE_GROUP \
  --name AppGW-WAF-Policy-$USER

• Creates an Application Gateway with the specified configurations, including the WAF policy, public IP, and subnet:

az network application-gateway create \
  --name "AppGW-aro-$USER" \
  --location $AZR_RESOURCE_LOCATION \
  --resource-group $AZR_RESOURCE_GROUP \
  --capacity 1 \
  --priority 1 \
  --sku WAF_v2 \
  --http-settings-cookie-based-affinity Disabled \
  --public-ip-address $AppGW_PIP \
  --vnet-name $ARO_VNET_NAME \
  --subnet $AppGW_SUBNET \
  --waf-policy AppGW-WAF-Policy-$USER

• The AppGW needs to be deployed and assigned to the proper resource group with the Public IP attached:

NOTE: The WAF policy needs to be enabled, because by default it’s in Disabled mode.

AppGW Load Balancer Application Certificates

The AppGW and our deployed apps will need to have the proper certificates attached, because the AppGW will also check against the backend whether the certificate presented is valid and has the proper FQDN.

• Generate SSL certificates using Let’s Encrypt’s Certbot tool:

export SCRATCH_DIR=/tmp/scratch
mkdir -p $SCRATCH_DIR

certbot certonly --manual \
  --preferred-challenges=dns \
  --email $EMAIL \
  --server https://acme-v02.api.letsencrypt.org/directory \
  --agree-tos \
  --manual-public-ip-logging-ok \
  -d "$APPGW_DOMAIN" \
  --config-dir "$SCRATCH_DIR/config" \
  --work-dir "$SCRATCH_DIR/work" \
  --logs-dir "$SCRATCH_DIR/logs"

NOTE: don’t close or interrupt this process, we will finish after the dns challenge in Azure.

• Open a second terminal and paste the DNS_Challenge (and remember to export the variables from the beginning again):

export DNS_CHALLENGE="xxxx"

• Adds a TXT record to the Azure DNS zone for the ACME challenge:

az network dns record-set txt add-record \
  --resource-group $AZR_DNS_RESOURCE_GROUP \
  --zone-name $DNS_ZONE_NAME \
  --record-set-name "_acme-challenge.$APP_NAME" \
  --value "$DNS_CHALLENGE"

• Wait up to 5 mins (maybe more) until the TXT record propagates and check the DNS resolution from the ACME challenge within Azure DNS. Check that the dig output matches the DNS Challenge shown earlier by the certbot command:

dig @8.8.8.8 +short TXT _acme-challenge.$APPGW_DOMAIN

@8.8.8.8 (not use the local dns cached)

• Return to the previous terminal and finish the generation of the ACME certificate for our ARO example App

• Certificate Bundle: Concatenate the generated SSL certificates into a bundle file and export it as a PKCS12 file.

export PFX_PASS="mypa55w0rd"

cat $SCRATCH_DIR/config/live/$APPGW_DOMAIN/fullchain.pem $SCRATCH_DIR/config/live/$APPGW_DOMAIN/privkey.pem > $SCRATCH_DIR/config/live/$APPGW_DOMAIN/gw-bundle.pem

openssl pkcs12 -export -out $SCRATCH_DIR/config/live/$APPGW_DOMAIN/gw-bundle.pfx -in $SCRATCH_DIR/config/live/$APPGW_DOMAIN/gw-bundle.pem

• Delete the TXT record created for the ACME challenge:

az network dns record-set txt delete \
  --resource-group $AZR_DNS_RESOURCE_GROUP \
  --zone-name $DNS_ZONE_NAME \
  --name "_acme-challenge.$APP_NAME"

Updating the DNS Records for AppGW and the exposed app

We need to update the DNS records for AppGW, using the Public IP that was generated in the step before.

• Retrieve the public IP address of the Application Gateway:

AGW_PIP=$(az network public-ip show -g $AZR_RESOURCE_GROUP --name $AppGW_PIP --query ipAddress -o tsv)

• Update the DNS record with the Application Gateway’s public IP address:

az network dns record-set a add-record \
--resource-group $AZR_DNS_RESOURCE_GROUP \
--zone-name $DNS_ZONE_NAME \
--record-set-name "$(echo $APPGW_DOMAIN | sed 's/\..*//')"  \
--ipv4-address $AGW_PIP

• Verify the DNS resolution for the updated domain:

dig @8.8.8.8 +short $APPGW_DOMAIN

• Creates an HTTPS listener for the Application Gateway:

az network application-gateway ssl-cert create \
  --resource-group $AZR_RESOURCE_GROUP \
  --gateway-name "AppGW-aro-$USER" \
  --name gw-bundle \
  --cert-file $SCRATCH_DIR/config/live/$APPGW_DOMAIN/gw-bundle.pfx \
  --cert-password $PFX_PASS

AppGW Listeners and Backends

• In the Listeners section, create a new HTTPS listener using the Azure portal:

Listener name: aro-route-https-listener
Frontend IP: Public
Port: 443
Protocol: HTTPS
Http Settings - choose to Upload a Certificate (upload the PFX file from earlier)
Cert Name: gw-bundle
PFX certificate file: gw-bundle.pfx
Host Type: single 
Host name: $APPGW_DOMAIN (aro-hello-openshift.test.openshiftdemo.dev)

Note: You can also create multiple listeners - one per site - and re-use the certificate and select basic site. We are using the Azure Portal here because the CLI doesn’t support MultiHostnames.

• Create a new backend pool (cli):

az network application-gateway address-pool create \
  --gateway-name "AppGW-aro-$USER" \
  --resource-group $AZR_RESOURCE_GROUP \
  --name aro-routes \
  --servers aro-hello-openshift.test.openshiftdemo.dev

• Create a new backend HTTP setting using the Azure Portal:

In the HTTP settings section, create a new HTTP setting:
HTTP settings name: aro-route-https-settings
Backend protocol: HTTPS
Backend port: 443
Use well known CA certificate: Yes (if you used one; otherwise upload your CA cer file)
Override with new host name: Yes
Choose: Override with specific domain name
Host name: $APPGW_DOMAIN

NOTE: We are using the Azure Portal because the CLI doesn’t support MultiHostnames.

• Define a rule for each website/api (cli):

az network application-gateway rule create \
  --gateway-name "AppGW-aro-$USER" \
  --resource-group $AZR_RESOURCE_GROUP \
  --name aro-app-https-rules \
  --http-listener aro-route-https-listener \
  --address-pool aro-routes \
  --http-settings aro-route-https-settings \
  --priority 2

Exposing HTTPD App

Now it’s time to publish our Hello OpenShift app deployed in the Private Cluster, exposed using the AppGW.

• Open an sshuttle connection to the ARO Private Cluster:

JUMP_IP=$(az vm list-ip-addresses -g $AZR_RESOURCE_GROUP -n aro-$USER-jumphost -o tsv \
--query '[].virtualMachine.network.publicIpAddresses[0].ipAddress')
echo $JUMP_IP

sshuttle --dns -NHr "aro@${JUMP_IP}"  10.0.0.0/8 --daemon

• Login to ARO Private Cluster (using sshuttle):

ARO_URL=$(az aro show -g $AZR_RESOURCE_GROUP -n $AZR_CLUSTER --query apiserverProfile.url -o tsv)
ARO_PASS=$(az aro list-credentials --name $AZR_CLUSTER --resource-group $AZR_RESOURCE_GROUP -o tsv --query kubeadminPassword)
oc login --username kubeadmin --password $ARO_PASS --server=$ARO_URL
ARO_DOMAIN=$(oc get dns cluster -o jsonpath='{.spec.baseDomain}')

• Create new project for testing app:

oc new-project aro-appgw

* Deploy an httpd server K8s Deployment and expose it using a K8s Service:

```md
oc create deployment hello-openshift --image=quay.io/openshifttest/hello-openshift:1.2.0 --port 8080
oc expose deployment hello-openshift

• Add the Edge Route with the hostname as the “aro-httpd.$DOMAIN”

oc create route edge --service=hello-openshift --hostname=$APPGW_DOMAIN \
    --key $SCRATCH_DIR/config/live/$APPGW_DOMAIN/privkey.pem \
    --cert $SCRATCH_DIR/config/live/$APPGW_DOMAIN/fullchain.pem

Testing that the App works

Now that we’ve deployed the App, let’s test if it works.

• Grab the App Route:

APP=$(oc get route hello-openshift -o jsonpath='{.spec.host}')

• Execute a couple of requests, and check the response code:

curl https://$APP
Hello OpenShift!

curl https://$APP -I
HTTP/1.1 200 OK
x-request-port: 8080

• Check the App exposed using the Custom Domain, and published in the AppGW Listener:

And with that, this blog post about exposing Applications using App Gateway Load Balancers in Private ARO clusters comes to a close.

NOTE: Opinions expressed in this blog are my own and do not necessarily reflect that of the company I work for.

Happy OpenShifting!

NOTE: All the code / examples used in this blog post are available in a GitHub repository. Check this out!

1. Gradio

A highly effective approach for showcasing your machine learning model, API, or data science workflow to others involves developing an interactive application that enables users or peers to experiment with the demo directly through their web browsers.

Gradio, a Python library, offers a streamlined solution for constructing such demos and facilitating easy sharing. In many cases, achieving this only requires a concise snippet of code.

1.1 Gradio and Kubernetes: the perfect match!

When coupled with Kubernetes, this approach gains additional advantages. Kubernetes provides a robust orchestration platform that enables seamless deployment, management, and scaling of containerized applications. By combining Gradio’s interactive demos with Kubernetes, you harness the power of containerization, making it easier to package and distribute your machine learning applications consistently across various environments.

The benefits of using Kubernetes alongside Gradio include:

• Scalability: Kubernetes allows your interactive demos built with Gradio to be easily scaled up or down based on demand. This ensures that as more users interact with your demos, the underlying infrastructure can handle the load efficiently.
• Resource Efficiency: Kubernetes optimizes resource utilization, ensuring that your demos running on Gradio are allocated the right amount of computational resources. This prevents overutilization or underutilization, leading to cost savings and better performance.
• High Availability: Kubernetes provides features like automatic load balancing and failover, which enhance the availability of your demos. This means that even if one instance fails, others will seamlessly take over, minimizing downtime.
• Monitoring and Management: Kubernetes offers robust monitoring and management tools. You can easily track the performance of your interactive demos, gather metrics, and troubleshoot any issues that arise.
• Consistency: Kubernetes ensures consistency across different deployment environments. Your Gradio-based demos will behave consistently whether they are running on your local machine, a development server, or a production cluster.
• Easy Updates and Rollbacks: Kubernetes facilitates smooth updates and rollbacks of your demos. This is crucial when you want to introduce new features or fixes without disrupting user interactions.

In summary, combining Gradio with Kubernetes not only allows you to create engaging interactive demos but also guarantees efficient deployment, scaling, monitoring, and management of these demos across different environments. This synergy empowers you to share your machine learning applications effectively and ensure a positive user experience.

2. Install K8s Cluster using KIND

Kind (Kubernetes in Docker) is a tool that allows you to create local Kubernetes clusters using Docker containers. It provides an environment to run Kubernetes clusters for development, testing, and experimentation purposes.

The benefits of using Kind include easy setup and teardown of clusters, fast cluster creation, and the ability to simulate multi-node Kubernetes clusters on a single machine. It helps streamline the development and testing workflow by providing a lightweight and isolated environment that closely resembles a production Kubernetes cluster.

• Install Docker and ensure that the Docker service is enabled and running:

sudo dnf config-manager --add-repo=https://download.docker.com/linux/centos/docker-ce.repo
sudo dnf install docker-ce --nobest 
sudo systemctl enable --now docker

NOTE: also Podman can be used, but for certain parts of this blog post, Docker worked out of the box without further tweaks in KIND.

• Create a Kind K8s cluster to deploy Seldon Core:

CLUSTER_NAME="k8s"
cat <<EOF | kind create cluster --name $CLUSTER_NAME --wait 200s --config=-
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
  kubeadmConfigPatches:
  - |
    kind: InitConfiguration
    nodeRegistration:
      kubeletExtraArgs:
        node-labels: "ingress-ready=true"
  extraPortMappings:
  - containerPort: 80
    hostPort: 80
    protocol: TCP
  - containerPort: 443
    hostPort: 443
    protocol: TCP
EOF

kubectl cluster-info --context kind-k8s

3. Installing K8s Ingress Controller

Kubernetes Ingress is crucial for managing external access to services within a cluster, providing routing and load balancing capabilities. Nginx Ingress, as a popular Ingress controller, enables seamless traffic distribution, SSL termination, and routing based on hostnames or paths, enhancing scalability and security.

• Install the Ingress Nginx adapted for Kind:

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/provider/kind/deploy.yaml

The provided command installs Nginx Ingress, extending Kubernetes functionality by efficiently directing incoming external requests to appropriate services using defined rules and configurations. This optimizes resource utilization and simplifies external connectivity management.

4. Visual Recognition Machine Learning model using MobileNetV2

MobileNetV2 is a convolutional neural network architecture that seeks to perform well on mobile devices. It is based on an inverted residual structure where the residual connections are between the bottleneck layers. The intermediate expansion layer uses lightweight depthwise convolutions to filter features as a source of non-linearity. As a whole, the architecture of MobileNetV2 contains the initial fully convolution layer with 32 filters, followed by 19 residual bottleneck layers.

You can find more information about MobileNetV2 in the official paper released by Sandler et al.

• Let’s deep dive a bit into the ML App code!

import requests
import tensorflow as tf
import gradio as gr

# load the model
mobile_net = tf.keras.applications.MobileNetV2()  

# Download human-readable labels for ImageNet.
response = requests.get("https://git.io/JJkYN")
labels = response.text.split("\n")

# Define a function classify_image(inp) that preprocesses input image, performs prediction using 
# inception_net, and returns a dictionary of class labels with corresponding probabilities.
def classify_image(input_images):
    input_images = input_images.reshape((-1, 224, 224, 3))
    input_images = tf.keras.applications.mobilenet_v2.preprocess_input(input_images)
    prediction = mobile_net.predict(input_images).flatten()
    return {labels[i]: float(prediction[i]) for i in range(1000)}

This code demonstrates the creation of an image classification system using TensorFlow and Gradio. It starts by importing necessary libraries: requests for HTTP requests, tensorflow for machine learning, and gradio for creating a user interface.

The MobileNetV2 model is loaded from tf.keras.applications and initialized. This model is a deep neural network pre-trained on a large dataset and capable of classifying images into numerous categories.

The code then fetches human-readable class labels for the ImageNet dataset, which contains over a thousand different object categories. These labels will be used to interpret the model’s predictions.

The classify_image function is defined to classify input images. It takes raw image data, reshapes it, and preprocesses using MobileNetV2’s preprocessing function. The model then predicts the class probabilities for each image, and the results are flattened into a list.

The function returns a dictionary containing class labels and their corresponding probabilities, with the keys being the class labels and the values being the prediction probabilities.

5. Integrating Gradio for deploying the Machine Learning App

• Let’s integrate the Gradio Python library to deploy our deep learning image classification model in an easy and visual way!

# Define a run function that sets up an image and label for classification using the gr.Interface.
def run():
  image = gr.Image(shape=(224, 224))
  label = gr.Label(num_top_classes=4)
  title = "Rcarrata's Image Classification Example"

  demo = gr.Interface(
      fn=classify_image, inputs=image, outputs=label, interpretation="default", title=title
  )

  demo.launch(server_name="0.0.0.0", server_port=7860)

The Gradio library is utilized to set up a web-based interface for the image classification system. Users can upload images through this interface, and the classify_image function processes them using the MobileNetV2 model.

The uploaded images are fed into the classify_image function, and the predictions are generated. The interface then displays the class labels along with their respective probabilities, allowing users to understand the model’s assessment of the uploaded images.

By integrating Gradio, this code enables easy interaction with the image classification model without requiring users to write code. It provides an accessible way for individuals to explore how the model categorizes different images and assesses its confidence in those classifications.

6. Containerizing the ML App

Now it’s time to containerize our Machine Learning Image Classification into a Container Image in order to deploy it in k8s / OpenShift.

• Essentially the Containerfile is like any other Python app, so it’s quite straightforward:

FROM python:3.9
WORKDIR /app
COPY ./requirements.txt /app/requirements.txt
RUN pip install --no-cache-dir -r /app/requirements.txt
COPY main.py /app
EXPOSE 7860
CMD ["python", "main.py"]

• We will use a Makefile that wraps docker/podman build, tag, and push to Quay.io:

make all

And voilà, we have our brand new ML Visual Classification App container Image stored in Quay.io ready to be deployed!

NOTE: remember that it’s a PoC and this Dockerfile can be improved in several ways! Use best practices!!

7. Deploying our ML Container Image into K8s

Now that we have our ML App Container Image ready to be deployed, let’s try it!

The app is composed of a standard k8s Deployment, a K8s Service, and the K8s Ingress.

To deploy the manifests in our K8s KIND server, we will use kustomize:

kubectl apply -k manifests/overlays/

• After a while, we can check the pod running:

kubectl get pod -n gradioapp
NAME                         READY   STATUS    RESTARTS   AGE
gradioapp-7fcf59fcb8-rw9pg   1/1     Running   0          22s

• If we check the Ingress, our app is exposed on port 80 using the Nginx Ingress:

kubectl get ingress -n gradioapp
NAME                CLASS    HOSTS   ADDRESS     PORTS   AGE
gradioapp-ingress   <none>   *       localhost   80      110s

Testing our Gradio App code with some examples

Now that it’s deployed, let’s have fun with our app!

• If we check the web browser localhost:80 app (in my case I deployed the KIND server on an external NUC server), we can see our brand new Gradio App:

Let’s test it with a couple of examples!

First let’s use an image of a German Shepherd (my favourite dog):

Pretty cool huh? In just a few seconds and with a relatively small Machine Learning model, we identified with 86% accuracy that it is a German Shepherd!

Let’s try another animal such as a Tiger:

And if we switch to a “thing”, like a fancy car (Ferrari), what will happen?

Also pretty accurate!

8. Interacting with our App Api using Gradio Client libraries

Another way to interact with our ML App is using Gradio Client, a Python library to handle requests to Gradio Apps.

I’ve written a small Python program to handle requests to the Gradio App deployed in k8s:

python test_app.py --url http://192.168.3.3 --image ./assets/tiger.jpeg
Loaded as API: http://192.168.3.3/ ✔
/var/folders/gc/9v6_6d8s2q51clcgwz747j2m0000gn/T/gradio/tmpcu35x_x8.json

We received the result of our requested prediction inference almost instantly from our App. If we check the results stored in JSON, we can see the exact same results as when we interacted with the browser:

cat /var/folders/gc/9v6_6d8s2q51clcgwz747j2m0000gn/T/gradio/tmpcu35x_x8.json | jq -r .
{
  "label": "tiger",
  "confidences": [
    {
      "label": "tiger",
      "confidence": 0.8418528437614441
    },
    {
      "label": "tiger cat",
      "confidence": 0.08996598422527313
    },
    {
      "label": "zebra",
      "confidence": 0.003402276895940304
    },
    {
      "label": "lynx",
      "confidence": 0.0014183077728375793
    }
  ]
}

And with that, this blog post about deploying and testing Machine Learning Applications in Kubernetes with Gradio comes to a close.

NOTE: Opinions expressed in this blog are my own and do not necessarily reflect that of the company I work for.

Happy MLOpsing!

1. Seldon Core Overview

Seldon Core is an open-source platform that helps data scientists and engineers deploy, scale, monitor, and manage machine learning models in Kubernetes. It is designed to wrap machine learning models and expose them as services that can be readily consumed by other applications.

Seldon Core offers a set of tools to build a machine learning model pipeline that can include feature extraction, outlier detection, model prediction, and many other components. It also provides the ability to deploy these pipelines in a distributed fashion and manage them using a unified interface. Seldon Core follows the Kubernetes philosophy of declarative definitions for all components.

Key features of Seldon Core include:

• Multiple language support: You can deploy models built in Python, R, Java, etc.
• Model versioning: Seldon Core can handle multiple versions of the same model for comparison or rollback purposes.
• Scalability: You can scale your deployments horizontally, as per the demand.
• Monitoring: Seldon Core provides tools to monitor your model’s performance and usage.
• Integration with popular ML libraries and frameworks: Supports various ML libraries including TensorFlow, PyTorch, XGBoost, and many more.

NOTE: In this blog post we are using a Baremetal Server with a CentOS 9 Stream OS and 64GB of RAM with 8 vCPUs (no GPU installed).

2. Install K8s Cluster using KIND

Kind (Kubernetes in Docker) is a tool that allows you to create local Kubernetes clusters using Docker containers. It provides an environment to run Kubernetes clusters for development, testing, and experimentation purposes.

The benefits of using Kind include easy setup and teardown of clusters, fast cluster creation, and the ability to simulate multi-node Kubernetes clusters on a single machine. It helps streamline the development and testing workflow by providing a lightweight and isolated environment that closely resembles a production Kubernetes cluster.

• Install Docker and ensure that the Docker service is enabled and running:

sudo dnf config-manager --add-repo=https://download.docker.com/linux/centos/docker-ce.repo
sudo dnf install docker-ce --nobest 
sudo systemctl enable --now docker

NOTE: also Podman can be used, but for certain parts of this blog post, Docker worked out of the box without further tweaks in KIND.

• Create a Kind K8s cluster to deploy Seldon Core:

CLUSTER_NAME="seldon"
cat <<EOF | kind create cluster --name $CLUSTER_NAME --wait 200s --config=-
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
  kubeadmConfigPatches:
  - |
    kind: InitConfiguration
    nodeRegistration:
      kubeletExtraArgs:
        node-labels: "ingress-ready=true"
  extraPortMappings:
  - containerPort: 80
    hostPort: 80
    protocol: TCP
  - containerPort: 443
    hostPort: 443
    protocol: TCP
EOF

kubectl cluster-info --context kind-seldon

3. Installing K8s Ingress Controller

Kubernetes Ingress is crucial for managing external access to services within a cluster, providing routing and load balancing capabilities. Nginx Ingress, as a popular Ingress controller, enables seamless traffic distribution, SSL termination, and routing based on hostnames or paths, enhancing scalability and security.

• Install the Ingress Nginx adapted for Kind:

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/provider/kind/deploy.yaml

The provided command installs Nginx Ingress, extending Kubernetes functionality by efficiently directing incoming external requests to appropriate services using defined rules and configurations. This optimizes resource utilization and simplifies external connectivity management.

4. MetalLB

Kubernetes lacks native support for network load balancers (LoadBalancer-type Services) in bare-metal clusters. The existing load balancer implementations in Kubernetes are essentially connectors to various IaaS platforms (GCP, AWS, Azure…). Because our setup doesn’t match these supported IaaS platforms, newly created LoadBalancers will indefinitely stay in a “pending” state.

Because of that, with our Baremetal K8s clusters we are left with two suboptimal options to direct user traffic to our apps in the K8s clusters: “NodePort” and “externalIPs” services. Both choices have notable drawbacks for production use, relegating Baremetal clusters to a secondary position in the Kubernetes ecosystem.

MetalLB seeks to rectify this situation by providing a network load balancer solution that seamlessly integrates with standard network equipment. This approach ensures that external services function as smoothly as possible on Baremetal clusters, addressing the existing imbalance.

4.1 Install MetalLB

• Since version 0.13.0, MetalLB is configured via CRs and the original way of configuring it via a ConfigMap based configuration is not working anymore:

kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.13.9/config/manifests/metallb-native.yaml

• Wait until the MetalLB pods (controller and speakers) are ready:

kubectl wait --namespace metallb-system \
                --for=condition=ready pod \
                --selector=app=metallb \
                --timeout=90s

4.2 Setup address pool used by MetalLB Load Balancers in KIND

With MetalLB, Layer 2 mode is the simplest for us to configure: in many cases, we don’t require any protocol-specific setup, only IP addresses.

In our Layer 2 mode, we don’t need the IPs to be tied to our worker nodes’ network interfaces. The system operates by directly responding to ARP requests on our local network, furnishing clients with the machine’s MAC address.

• To finalize the layer2 setup, we must provide MetalLB with a designated IP address range under its control. Our intention is for this range to be within the Docker Kind network:

docker network inspect -f '' kind

NOTE: When using Docker on Linux (or KIND), it’s possible to route traffic directly to the external IP of the load balancer, given that the IP range falls within the Docker IP space.

• The result will include a CIDR, like 172.19.0.0/16. Our aim is to allocate load balancer IP addresses from this specific subset. We can set up MetalLB, for example, to utilize the range from 172.19.255.200 to 172.19.255.250. This involves establishing an IPAddressPool and the associated L2Advertisement:

kubectl apply -f - << END
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: example
  namespace: metallb-system
spec:
  addresses:
  - 172.18.255.200-172.18.255.250
END

• To promote the IP originating from an IPAddressPool, an L2Advertisement instance needs to be linked with the respective IPAddressPool:

kubectl apply -f - << END
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  name: empty
  namespace: metallb-system
END

Setting no IPAddressPool selector in an L2Advertisement instance is interpreted as that instance being associated with all the available IPAddressPools.

4.3 Testing the MetalLB deployment

• To test our setup, we will deploy a dummy app and check if we can use the K8s LoadBalancer powered by MetalLB to access our app:

kubectl apply -f https://kind.sigs.k8s.io/examples/loadbalancer/usage.yaml
LB_IP=$(kubectl get svc/foo-service -o=jsonpath='{.status.loadBalancer.ingress[0].ip}')
curl ${LB_IP}:5678

5. ServiceMesh and Istio

A Service Mesh is an infrastructure layer added to modern distributed microservices applications, enhancing them with transparent capabilities like observability, traffic management, and security. It simplifies complex operational needs such as A/B testing, canary deployments, and access control.

Istio is an open source service mesh solution that seamlessly integrates with existing distributed applications. It provides centralized features like secure communication, load balancing, traffic control, access policies, and automatic metrics. Istio is adaptable, supporting Kubernetes deployments and extending to other clusters or endpoints.

Its control plane offers TLS encryption, strong authentication, load balancing, and fine-grained traffic control. Istio’s ecosystem includes diverse contributors, partners, and integrations, making it versatile for various use cases.

We need Istio in order to deploy Seldon Core because it uses some functionality under the hood to deploy the ML models.

Let’s install Istio in Kind!

5.1 Install Istio in KIND

• Download and install Istioctl latest version:

curl -L https://istio.io/downloadIstio | sh -
cd istio-1.17.2
chmod u+x istioctl
cp -pr istioctl /usr/local/bin/

• Install Istio in our K8s cluster using istioctl:

istioctl install --set profile=demo -y

kubectl get service -n istio-system istio-ingressgateway

• Deploy the Bookinfo application to test the Service Mesh:

kubectl apply -f https://raw.githubusercontent.com/istio/istio/release-1.17/samples/bookinfo/platform/kube/bookinfo.yaml

kubectl apply -f https://raw.githubusercontent.com/istio/istio/release-1.17/samples/bookinfo/networking/bookinfo-gateway.yaml

• Retrieve and export the IP address of the Istio Ingress Gateway and the associated ports for HTTP and HTTPS services from the Kubernetes cluster’s Istio system namespace:

export INGRESS_HOST=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
export INGRESS_PORT=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="http2")].port}')
export SECURE_INGRESS_PORT=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="https")].port}')

• Test the Bookinfo ProductPage app using the Istio Ingress Gateway:

export GATEWAY_URL=$INGRESS_HOST:$INGRESS_PORT
curl -I http://$GATEWAY_URL/productpage

6. Seldon Core

Seldon Core converts your machine learning models (like TensorFlow, PyTorch, H2O, etc.) or language wrappers (Python, Java, etc.) into operational microservices for production, which use REST/gRPC.

Seldon takes care of expanding to numerous production-level machine learning models and offers advanced machine learning features right from the start. This includes advanced metrics, keeping track of requests, explanation tools, spotting outliers, A/B testing, canary deployments, and more.

6.1 Seldon Core install

• Install the Seldon Controller using Helm to manage your Seldon Deployment graphs:

kubectl create namespace seldon-system
helm install seldon-core seldon-core-operator \
    --repo https://storage.googleapis.com/seldon-charts \
    --set usageMetrics.enabled=true \
    --set istio.enabled=true \
    --namespace seldon-system

NOTE: seldon-system namespace is preferred and we are using the istio enabled because we will use Istio alongside Seldon to expose our models to the final users.

• Define the Istio Ingress Gateway for Seldon Core:

kubectl apply -f - << END
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: seldon-gateway
  namespace: istio-system
spec:
  selector:
    istio: ingressgateway # use istio default controller
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "*"
END

NOTE: this is just for a PoC, in production please use HTTPS/TLS instead of plain HTTP!

6.2 Seldon Core Workflow

Once we have installed Seldon Core, we can productize our model with the following three steps:

• Wrap our model using our prepackaged inference servers or language wrappers
• Define and deploy the Seldon Core inference graph
• Send predictions and monitor performance

Source - Seldon Core Inference Pipeline Documentation

6.2.1 Wrap our model using our prepackaged inference servers or language wrappers

To prepare components for production, we need to package them as Linux containers following the Seldon microservice API guidelines. These encompass prediction-serving models, decision-making routers like A-B Tests, response-combining Combiners, and versatile transformers for request/response modification.

To simplify the integration of machine learning components developed in diverse languages and toolkits, Seldon Core offers wrappers. These enable effortless creation of Docker containers from our code, suitable for execution within Seldon Core. The presently recommended tool for this purpose is Red Hat’s Source-to-Image.

6.2.2 Define and deploy the Seldon Core inference graph

Deploying our models using Seldon Core is simplified through Seldon pre-packaged inference servers and language wrappers, or by building our own Seldon Inference Graphs.

In this case we will use the Seldon prebuilt Sklearn Server, but there are many more prebuilt servers such as TensorFlow, Hugging Face, and other Seldon Inference Servers.

We can deploy our model by loading the binaries/artifacts using the pre-packaged model server of our choice. We can also build complex inference graphs that use multiple components for inference if needed.

To run our machine learning graph on Kubernetes we need to define how the components we created in the last step fit together to represent a service graph. This is defined inside a SeldonDeployment Kubernetes Custom resource.

• Let’s generate a SeldonDeployment in our Kubernetes cluster to deploy an example of the Sklearn Iris Model example, using the SKlearn Inference Server:

kubectl create namespace seldon

kubectl apply -f - << END
apiVersion: machinelearning.seldon.io/v1
kind: SeldonDeployment
metadata:
  name: iris-model
  namespace: seldon
spec:
  name: iris
  predictors:
  - graph:
      implementation: SKLEARN_SERVER
      modelUri: gs://seldon-models/v1.17.0-dev/sklearn/iris
      name: classifier
    name: default
    replicas: 1
END

6.2.3 Send a request to our Machine Learning model deployed using SeldonDeployment

Every deployed model exposes a standardized User Interface to send requests using the OpenAPI schema.

• Let’s send a request to our ML model deployed:

curl -X POST http://$GATEWAY_URL/seldon/seldon/iris-model/api/v1.0/predictions \
    -H 'Content-Type: application/json' \
    -d '{ "data": { "ndarray": [[5.964, 4.006, 2.081, 1.031]] } }'

The data includes a “data” field containing an array (ndarray) of inputs. In this case, a single set of input values is provided: [5.964, 4.006, 2.081, 1.031], representing features for making a prediction.

• We receive a response from our model inference API with the generated predictions:

{
   "meta" : {},
   "data" : {
      "names" : [
         "t:0",
         "t:1",
         "t:2"
      ],
      "ndarray" : [
         [
            0.000698519453116284,
            0.00366803903943576,
            0.995633441507448
         ]
      ]
   }
}

Now that we know how to use SeldonDeployment to deploy our Machine Learning models using Seldon’s prebuilt Inference Servers, we can test other Inference Servers such as TensorFlow Server.

7. Seldon TensorFlow MNIST Model

If we have a trained TensorFlow model, we can deploy it directly via REST or gRPC servers.

• Let’s deploy the Tensorflow MNIST Keras example with Tensorflow Server:

kubectl apply -f - << END
apiVersion: machinelearning.seldon.io/v1alpha2
kind: SeldonDeployment
metadata:
  name: tfserving
spec:
  name: mnist
  predictors:
  - graph:
      children: []
      implementation: TENSORFLOW_SERVER
      modelUri: gs://seldon-models/tfserving/mnist-model
      name: mnist-model
      parameters:
        - name: signature_name
          type: STRING
          value: predict_images
        - name: model_name
          type: STRING
          value: mnist-model
        - name: model_input
          type: STRING
          value: images
        - name: model_output
          type: STRING
          value: scores
    name: default
    replicas: 1
END

• We wait until the SeldonDeployment is up and running and ready to serve prediction requests:

kubectl rollout status deploy/$(kubectl get deploy -l seldon-deployment-id=tfserving -o jsonpath='{.items[0].metadata.name}')

• In this case, to test our deployed ML model, we will use the Seldon-Core Python libraries to request predictions. Let’s install the Python libraries on our system:

pip3 install setuptools-rust
pip3 install --upgrade pip
pip3 install seldon-core --ignore-installed PyYAML

• Once the libraries are installed, we can use the SeldonClient class to request a prediction from our deployed ML model Inference Server (exposed using the Istio Ingress Gateway):

from seldon_core.seldon_client import SeldonClient
sc = SeldonClient(deployment_name="tfserving", namespace="seldon")
r = sc.predict(gateway="istio", transport="rest", shape=(1, 784))
print(r)
assert r.success == True

• After that, our MNIST ML model Inference server answers with the predictions:

Success:True message:
Request:
meta {
}
data {
  tensor {
    shape: 1
    shape: 784
    values: 0.4689572966007861
    values: 0.9660213976358323
    values: 0.2439077409486442
    values: 0.8575884865204007
    values: 0.27970466773693103
...

And with that, this blog post about how to deploy Machine Learning Models using Seldon Core, MetalLB, and Istio in our KIND K8s clusters deployed on a Baremetal server comes to a close.

In upcoming blog posts we will analyze how we can deploy our ML models in an easy and scalable way in the cloud, using Seldon-Core, OpenShift, and OpenDataHub!

NOTE: Opinions expressed in this blog are my own and do not necessarily reflect that of the company I work for.

Happy MLOpsing!

Let’s dig in!

Overview

By default Azure Red Hat OpenShift uses self-signed certificates for all of the routes created on “*.apps.$random.$location.aroapp.io.”

Many companies also seek to leverage the capabilities of Azure Red Hat OpenShift (ARO) to deploy their applications while using their own custom domain. ARO offers the flexibility to integrate custom domains seamlessly, allowing organizations to align their cloud-based applications with their existing domain structure.

By utilizing ARO’s custom domain feature, companies can ensure a consistent branding experience by hosting their applications under their own domain name. This enables them to maintain brand recognition and create a cohesive user experience across various online touchpoints.

If we choose to specify a custom domain, for example aro.myorg.com, the OpenShift console will be available at a URL such as “https://console-openshift-console.apps.aro.myorg.com”, instead of the built-in domain “https://console-openshift-console.apps.$random.$location.aroapp.io.”

Furthermore, if we choose Custom DNS, after connecting to the cluster, we will need to configure a custom certificate for our ARO ingress controller and custom certificate of our API server.

1. Deploying ARO Prerequisites

Before deploying Azure Red Hat OpenShift (ARO), there are certain prerequisites that need to be fulfilled.

1.1 Variables and Resource Group

• Set the following environment variables:

export AZR_PULL_SECRET=~/Downloads/pull-secret.txt
export NETWORK_SUBNET=10.0.0.0/20
export CONTROL_SUBNET=10.0.0.0/24
export MACHINE_SUBNET=10.0.1.0/24
export JUMPHOST_SUBNET=10.0.3.0/24
export NAMESPACE=aro-custom-domain
export AZR_CLUSTER=aro-$USER
export AZR_RESOURCE_LOCATION=eastus
export AZR_RESOURCE_GROUP=aro-$USER-rg
export DOMAIN="aroplay.openshiftdemo.dev"
export AZR_DNS_RESOURCE_GROUP="mobb-dns"
export EMAIL=username.taken@gmail.com

• Create an Azure resource group:

az group create                \
  --name $AZR_RESOURCE_GROUP   \
  --location $AZR_RESOURCE_LOCATION

A resource group in Azure serves as a logical container for deploying and managing Azure resources. When creating a resource group, a specific location is required to store its metadata and determine the default region for resource deployment, unless otherwise specified during resource creation. Additionally, this chosen location defines the region in which the resources within the resource group will run in Azure.

1.2 ARO Networking prerequisites

To successfully run Azure Red Hat OpenShift clusters on OpenShift 4, it is necessary to have a virtual network with two empty subnets specifically designated for the master and worker nodes.

Within the resource group you previously established, we need to create a new virtual network that will accommodate these requirements.

• Create virtual network:

az network vnet create                                    \
  --address-prefixes $NETWORK_SUBNET                      \
  --name "$AZR_CLUSTER-aro-vnet-$AZR_RESOURCE_LOCATION"   \
  --resource-group $AZR_RESOURCE_GROUP

• Create control plane subnet:

az network vnet subnet create                                     \
  --resource-group $AZR_RESOURCE_GROUP                            \
  --vnet-name "$AZR_CLUSTER-aro-vnet-$AZR_RESOURCE_LOCATION"      \
  --name "$AZR_CLUSTER-aro-control-subnet-$AZR_RESOURCE_LOCATION" \
  --address-prefixes $CONTROL_SUBNET                              \
  --service-endpoints Microsoft.ContainerRegistry

• Create machine subnet:

az network vnet subnet create                                       \
  --resource-group $AZR_RESOURCE_GROUP                              \
  --vnet-name "$AZR_CLUSTER-aro-vnet-$AZR_RESOURCE_LOCATION"        \
  --name "$AZR_CLUSTER-aro-machine-subnet-$AZR_RESOURCE_LOCATION"   \
  --address-prefixes $MACHINE_SUBNET                                \
  --service-endpoints Microsoft.ContainerRegistry

• Disable network policies for Private Link Service on the control plane subnet:

az network vnet subnet update                                       \
  --name "$AZR_CLUSTER-aro-control-subnet-$AZR_RESOURCE_LOCATION"   \
  --resource-group $AZR_RESOURCE_GROUP                              \
  --vnet-name "$AZR_CLUSTER-aro-vnet-$AZR_RESOURCE_LOCATION"        \
  --disable-private-link-service-network-policies true

1.3 Creating Private ARO Clusters with Custom Domain:

Creating Private Azure Red Hat OpenShift (ARO) clusters with a custom domain offers organizations the ability to establish a fully private environment for their applications. By configuring the cluster with private visibility for the API server and ingress, organizations ensure that all communication and access to the cluster remains within their private network, enhancing security and control over their infrastructure.

This enables organizations to create an isolated and customized environment for their applications, providing an elevated level of privacy and data protection.

• Create private ARO Cluster with Custom Domain:

az aro create \
--resource-group $AZR_RESOURCE_GROUP \
--name $AZR_CLUSTER \
--vnet "$AZR_CLUSTER-aro-vnet-$AZR_RESOURCE_LOCATION" \
--master-subnet "$AZR_CLUSTER-aro-control-subnet-$AZR_RESOURCE_LOCATION" \
--worker-subnet "$AZR_CLUSTER-aro-machine-subnet-$AZR_RESOURCE_LOCATION" \
--apiserver-visibility Private \
--ingress-visibility Private \
--pull-secret @$AZR_PULL_SECRET \
--domain $DOMAIN

When the –domain flag with an FQDN (e.g. my.domain.com) is used to create your cluster, we will need to configure DNS and a certificate authority for your API server and apps ingress.

1.4 Jumphost

As the cluster operates within a private network, it is possible to create a Jump host during the cluster creation process. This Jump host serves as a secure gateway that allows authorized users to connect to the private cluster environment.

• Create Jumphost subnet:

az network vnet subnet create                                \
  --resource-group $AZR_RESOURCE_GROUP                       \
  --vnet-name "$AZR_CLUSTER-aro-vnet-$AZR_RESOURCE_LOCATION" \
  --name JumpSubnet                                          \
  --address-prefixes $JUMPHOST_SUBNET                        \
  --service-endpoints Microsoft.ContainerRegistry

• Create a JumpHost:

az vm create --name jumphost                 \
    --resource-group $AZR_RESOURCE_GROUP     \
    --ssh-key-values $HOME/.ssh/id_rsa.pub   \
    --admin-username aro                     \
    --image "RedHat:RHEL:9_1:9.1.2022112113" \
    --subnet JumpSubnet                      \
    --public-ip-address jumphost-ip          \
    --public-ip-sku Standard                 \
    --vnet-name "$AZR_CLUSTER-aro-vnet-$AZR_RESOURCE_LOCATION"

• Save the jump host public IP address:

JUMP_IP=$(az vm list-ip-addresses -g $AZR_RESOURCE_GROUP -n jumphost -o tsv \
--query '[].virtualMachine.network.publicIpAddresses[0].ipAddress')
echo $JUMP_IP

• Use sshuttle to create an SSH VPN via the jump host as a daemon:

sshuttle --dns -NHr "aro@${JUMP_IP}"  10.0.0.0/8 --daemon

NOTE: While creating a Jump host during cluster creation provides a means for secure remote access, it is recommended to utilize a VPN (Virtual Private Network) or ExpressRoute for even stronger network connectivity and enhanced security. These solutions establish a secure and private connection between the on-premises network or other trusted networks and the private Azure Red Hat OpenShift (ARO) cluster, ensuring a robust and reliable communication channel.

2. Configure DNS for the Private ARO Cluster (Ingress Router and API)

Properly configuring DNS for the default ingress router, API server endpoint, and associated routes such as the console and *.apps is of utmost importance.

These DNS configurations ensure easy access to the cluster’s console, application routes, and APIs, facilitating smooth administration and interaction with the OpenShift/Kubernetes environment.

2.1 Configure DNS for default ingress router

We need to configure the DNS for the Default Ingress Router (*.apps) to be able to access the ARO Console, among other things.

• Retrieve the Ingress IP for Azure DNS records:

INGRESS_IP="$(az aro show -n $AZR_CLUSTER -g $AZR_RESOURCE_GROUP --query 'ingressProfiles[0].ip' -o tsv)"

echo $INGRESS_IP

2.1.1 Apps/Console Public Zone Ingress Configuration

• Create your Azure DNS zone for $DOMAIN:

az network dns zone create -g $RESOURCEGROUP -n $DOMAIN

az network dns zone create --parent-name $DOMAIN -g $AZR_DNS_RESOURCE_GROUP -n $DOMAIN

NOTE: Or use an existing zone if it exists. You need to have configured your domain name registrar to point to this zone.

• Add a record type A pointing the “*.apps.DOMAIN” to the Ingress LB IP, that is the Azure LB that balances the ARO/OpenShift Routers (Haproxies):

az network dns record-set a add-record \
  -g $AZR_DNS_RESOURCE_GROUP \
  -z $DOMAIN \
  -n '*.apps' \
  -a $INGRESS_IP

• Adjust default TTL from 1 hour (choose an appropriate value, here 5 mins is used):

az network dns record-set a update -g $AZR_DNS_RESOURCE_GROUP -z $DOMAIN -n '*.apps' --set ttl=300

• Test the *.apps domain:

dig +short test.apps.$DOMAIN

2.2 Configure DNS for API server endpoint

We need to configure the DNS for the Kubernetes / OpenShift API of the ARO cluster to be able to access the ARO API.

• Retrieve the API Server IP for Azure DNS records:

API_SERVER_IP="$(az aro show -n $AZR_CLUSTER -g $AZR_RESOURCE_GROUP --query 'apiserverProfile.ip' -o tsv)"
echo $API_SERVER_IP

• Create an api A record to point to the Ingress Load Balancer IP:

az network dns record-set a add-record \
  -g $AZR_DNS_RESOURCE_GROUP \
  -z $DOMAIN \
  -n 'api' \
  -a $API_SERVER_IP

• Optional (good for initial testing): Adjust default TTL from 1 hour (choose an appropriate value, here 5 mins is used):

az network dns record-set a update \
  -g $AZR_DNS_RESOURCE_GROUP \
  -z $DOMAIN \
  -n 'api' \
  --set ttl=300

• Test the api domain:

dig +short api.$DOMAIN

NOTE: In our scenario, the Jumphost will be used for connecting to the cluster via both console and API. Since we are utilizing various subnets within the same VNet, there’s no need to generate a Private Zone to resolve DNS entries from the Jumphost.

However, if you are dividing the Bastion/Jumphost across different VNets, you may need to create an Azure Private Zone and the Private Link.

3. Generate Let’s Encrypt Certificates for API Server and default Ingress Router

The following example employs manually created Let’s Encrypt certificates. However, it’s important to note that this is not recommended for production environments unless an automated process has been established for the generation and renewal of these certificates (for instance, through the use of the Cert-Manager operator).

Keep in mind that these certificates are subject to expiry after 90 days.

NOTE: this method relies on public DNS for the issuance of certificates since it uses a DNS challenge. Once the certificates have been issued, if desired, the public records can be removed (this could be the case if you’ve created a private ARO cluster and plan to use Azure DNS private record sets).

3.1 Generate LE Certs for default Ingress Router (*.apps/console)

• Create TLS Key Pair for the apps/console domain using certbot:

export SCRATCH_DIR=/tmp/scratch

certbot certonly --manual \
  --preferred-challenges=dns \
  --email $EMAIL \
  --server https://acme-v02.api.letsencrypt.org/directory \
  --agree-tos \
  --config-dir "$SCRATCH_DIR/config" \
  --work-dir "$SCRATCH_DIR/work" \
  --logs-dir "$SCRATCH_DIR/logs" \
  -d "*.apps.$DOMAIN"

• Take note of the Domain and TXT value fields as these are required for Let’s Encrypt to validate that you own the domain and can therefore issue you the certificates.

NOTE: don’t close or interrupt this process, we will finish after the dns challenge with.

• Open a second terminal and paste the DNS_Challenge (and remember to export again the variables from the beginning):

export APPS_TXT_RECORD="xxxx"

• You can add the necessary records to validate ownership of the apps domain:

az network dns record-set txt add-record \
  -g $AZR_DNS_RESOURCE_GROUP \
  -z $DOMAIN \
  -n "_acme-challenge.apps" \
  -v $APPS_TXT_RECORD

• Update the TTL for the records from 1h to 5 minutes for testing purposes:

az network dns record-set txt update \
  -g $AZR_DNS_RESOURCE_GROUP \
  -z $DOMAIN \
  -n "_acme-challenge.apps" \
  --set ttl=300

• Make sure that the TXT record from the Azure domain challenge is registered and propagated properly:

dig +short TXT _acme-challenge.apps.$DOMAIN

• Return to the first terminal (where the certbot is), and finish the generation of the apps certificate PKIs for the ARO cluster.

3.2 Generate LE Certs for the api

• Create TLS Key Pair for the api domain using certbot:

export SCRATCH_DIR=/tmp/scratch

certbot certonly --manual \
  --preferred-challenges=dns \
  --email $EMAIL \
  --server https://acme-v02.api.letsencrypt.org/directory \
  --agree-tos \
  --config-dir "$SCRATCH_DIR/config" \
  --work-dir "$SCRATCH_DIR/work" \
  --logs-dir "$SCRATCH_DIR/logs" \
  -d "api.$DOMAIN"

NOTE: don’t close or interrupt this process, we will finish after the dns challenge with the certbot.

• Open a second terminal and paste the DNS_Challenge (and remember to export again the variables from the beginning):

export API_TXT_RECORD="xxxx"

• You can add the necessary records to validate ownership of the api domain:

az network dns record-set txt add-record \
  -g $AZR_DNS_RESOURCE_GROUP \
  -z $DOMAIN \
  -n "_acme-challenge.api" \
  -v $API_TXT_RECORD

• Adjust default TTL from 1 hour (choose an appropriate value, here 5 mins is used):

az network dns record-set txt update \
  -g $AZR_DNS_RESOURCE_GROUP \
  -z $DOMAIN \
  -n "_acme-challenge.api" \
  --set ttl=300

• Make sure that the TXT record from the Azure domain challenge is registered and propagated properly:

dig +short TXT _acme-challenge.api.$DOMAIN

• Return to the first terminal (where the certbot is), and finish the generation of the API certificate PKIs for the ARO cluster.

4.1 Configure the Ingress Router with custom certificates

By default, the OpenShift Container Platform uses the Ingress Operator to generate an internal Certificate Authority (CA) and issue a wildcard certificate, which is valid for applications under the .apps sub-domain. This certificate is used by both the web console and CLI.

You can replace the default ingress certificate for all applications under the .apps subdomain. After you replace the certificate, all applications, including the web console and CLI, will have encryption provided by specified certificate.

• Configure the API server with custom certificates:

AROPASS=$(az aro list-credentials --name $AZR_CLUSTER --resource-group $AZR_RESOURCE_GROUP -o tsv --query kubeadminPassword)
AROURL=$(az aro show -g $AZR_RESOURCE_GROUP -n $AZR_CLUSTER --query apiserverProfile.url -o tsv)

• Login to the ARO cluster with oc CLI:

oc login -u kubeadmin -p $AROPASS --server=$AROURL --insecure-skip-tls-verify=true

Please note that we are currently utilizing the “–insecure-skip-tls-verify=true” flag due to the presence of self-signed certificates in both the API and the default ingress controller (*.apps).

• Create a config map that includes only the root CA certificate used to sign the wildcard certificate:

oc create configmap custom-ca \
     --from-file=$SCRATCH_DIR/config/live/apps.$DOMAIN/fullchain.pem \
     -n openshift-config

• Update the cluster-wide proxy configuration with the newly created config map:

oc patch proxy/cluster \
     --type=merge \
     --patch='{"spec":{"trustedCA":{"name":"custom-ca"}}}'

• Create a secret that contains the wildcard certificate chain and key:

oc create secret tls apps-custom-domain \
     --cert=$SCRATCH_DIR/config/live/apps.$DOMAIN/fullchain.pem \
     --key=$SCRATCH_DIR/config/live/apps.$DOMAIN/privkey.pem \
     -n openshift-ingress

• Update the Ingress Controller configuration with the newly created secret:

oc patch ingresscontroller.operator default \
--type=merge -p \
'{"spec":{"defaultCertificate":{"name":"apps-custom-domain"}}}' \
-n openshift-ingress-operator

• Check the OpenShift Ingress pods:

  oc get pod -n openshift-ingress
  

• Verify that your certificate is correctly applied:

echo | openssl s_client -connect console-openshift-console.apps.$DOMAIN:443 | openssl x509 -noout -text | grep Issuer

• Check that the certificate when you access the Console is the one issued by Let’s Encrypt using Certbot:

4.2 Configure the API server with custom certificates

• Create a secret that contains the certificate chain and private key in the openshift-config namespace:

oc create secret tls api-custom-domain-cert \
     --cert=$SCRATCH_DIR/config/live/api.$DOMAIN/fullchain.pem \
     --key=$SCRATCH_DIR/config/live/api.$DOMAIN/privkey.pem \
     -n openshift-config

• Update the API server certificate to reference the created secret. Patch the cluster’s API server and **replace with your customer domain**:

oc patch apiserver cluster \
--type=merge -p \
'{"spec":{"servingCerts":{"namedCertificates":
[{"names":["api.<DOMAIN>"],
"servingCertificate":{"name":"api-custom-domain-cert"}}]}}}'

• Check the apiserver cluster CRD to check if the patch worked properly:

oc get apiserver cluster -o yaml

• After a couple of minutes, check the certificate exposed:

echo | openssl s_client -connect api.$DOMAIN:6443 | openssl x509 -noout -text | grep Issuer

• Logout and login without the “–insecure-skip-tls-verify=true”:

oc logout
oc login -u kubeadmin -p $AROPASS --server=$AROURL

And with that, this blog post about how to create Private ARO clusters with Custom Domain comes to a close.

NOTE: Opinions expressed in this blog are my own and do not necessarily reflect that of the company I work for.

Happy DNSing!

As we explained in our first blog post, software supply chain is the series of steps performed when writing, testing, packaging, and distributing application software to end consumers.

Establishing trust in the software supply chain has become essential to ensuring the security and reliability of software components. With the majority of software being open-source and the growing demand for supply chain management, it’s crucial to have robust processes to prevent malicious code or dependencies from being introduced; integrating open-source tools can enhance the software supply chain’s lifecycle security.

In this blog post that my friend and colleague Rodrigo Alvares prepared, we will discuss the actions you can take to establish trust in the software supply chain and the critical role that open-source tools can play in verifying the reliability of software components.

NOTE: this blog post was originally posted on Opensourcerers on the 24th of April 2023.

1. Distributed components of the Secure Software Factory

The software supply chain is a crucial process that involves multiple steps, including writing, testing, packaging, and delivering application software to end-users. With the growing occurrence of software supply chain exploits and attacks, the Cloud Native Computing Foundation (CNCF) Technical Advisory Group for Security has taken proactive steps by publishing a comprehensive whitepaper titled “Software Supply Chain Best Practices” adopting the Software Factory Model for designing a secure software supply chain: The Secure Software Factory.

The Secure Software Factory relies on source code, which includes the human-readable representation of applications being developed, as well as any dependencies that are either built from source or interpreted instead of compiled. The source code for both the build pipelines (Pipeline-as-Code) and the infrastructure (Infrastructure-as-Code) are included.

During the pipeline’s execution, various metadata documents are created, including test reports, vulnerability reports, software attestations and Software Bills of Material (SBOMs). These documents capture the state of the build that generated them.

For instance, a vulnerability report includes CVEs that were known during the build, but its accuracy may decline as new vulnerabilities are identified and disclosed. Similarly, an SBOM represents the contents of a specific build and remains relevant for that build. However, if future builds have slightly different dependencies or version constraints, a new or updated SBOM must be generated.

A software attestation refers to a verified metadata statement regarding a software artifact or a set of artifacts. Its main purpose is to provide input to automated policy engines, like Binary Authorization and in-toto.

The SSF assumes that source code is managed using version control systems such as Git, with an established review and testing process in place that is suitable for the repository’s needs and use cases.

As the primary input for the SSF, it is up to the users and operators to determine which programming languages to support, where to host the source code, and which testing and scanning tools to integrate.

In a nutshell, we need to be able to check the origin of all of these artifacts, like source code and dependencies (among others like attestations, signatures, etc.) and to trace all the packages and artifacts of the components of our Software Supply Chain.

2. Secure Software Factory Artifacts

The output that the Secure Software Factory produces is known as a software artifact, which serves as the primary deliverable. This artifact can take various forms, such as binaries, software packages, container images, signatures, or attestations, and it is designed to be utilized by downstream users.

To validate the artifact’s origin, it must be accompanied by the appropriate metadata, securely stored in an artifact repository, and distributed using secure and well-understood channels.

The specifics of the artifact’s characteristics and the execution of these requirements may differ based on variables like a programming language, package type, and target platform(s). Therefore, the Secure Software Factory does not address these implementation details.

Let’s see how we can use Open Source tooling to generate and manage all of these distributed components for our Secure Software Factory Supply Chain.

2.1 Building an example Application Artifact

Petclinic is a Spring Boot application built using Maven or Gradle, that we will use as an example in this blog post. We will download the source code and then build the Java application artifacts (JARs) that will be used for running this application:

$ git clone https://github.com/spring-projects/spring-petclinic.git
$ cd spring-petclinic
$ ./mvnw package
$ java -jar target/*.jar

Some of the best practices for the Artifacts are the following:

• Artifacts must be available to downstream consumers and securely stored.
• Signatures for artifacts should also be stored such that they can easily be found and verified.
• These signatures can be stored alongside the artifact for convenient discoverability and distribution or in a separate location.

2.2 Dependency-Check (SCA)

Dependency-Check is a Software Composition Analysis (SCA) tool that attempts to detect publicly disclosed vulnerabilities contained within a project’s dependencies. It does this by determining if there is a Common Platform Enumeration (CPE) identifier for a given dependency. If found, it will generate a report linking to the associated CVE entries.

After installing locally, we can run a dependency check in our folder where we downloaded the source code and built all the artifacts, exploring the project dependencies and generating a report:

$ dependency-check –enableExperimental –scan .
[INFO] Checking for updates
[INFO] NVD CVE requires several updates; this could take a couple of minutes.
[INFO] Download Started for NVD CVE – 2002
[INFO] Download Complete for NVD CVE – 2002  (1117 ms)
[INFO] Processing Started for NVD CVE – 2002
[INFO] Processing Complete for NVD CVE – 2002  (3210 ms)
…
[INFO] Analysis Started
[INFO] Finished Archive Analyzer (2 seconds)
[INFO] Finished File Name Analyzer (0 seconds)
[INFO] Finished Jar Analyzer (0 seconds)
[INFO] Finished Central Analyzer (32 seconds)
[INFO] Finished Python Distribution Analyzer (0 seconds)
[INFO] Finished Node.js Package Analyzer (0 seconds)
[INFO] Finished Dependency Merging Analyzer (0 seconds)
[INFO] Finished Version Filter Analyzer (0 seconds)
[INFO] Finished Hint Analyzer (0 seconds)
[INFO] Created CPE Index (0 seconds)
[INFO] Finished NPM CPE Analyzer (1 seconds)
[INFO] Created CPE Index (0 seconds)
[INFO] Finished CPE Analyzer (1 seconds)
[INFO] Finished False Positive Analyzer (0 seconds)
[INFO] Finished NVD CVE Analyzer (0 seconds)
[INFO] Finished RetireJS Analyzer (0 seconds)
[INFO] Finished Sonatype OSS Index Analyzer (1 seconds)
[INFO] Finished Vulnerability Suppression Analyzer (0 seconds)
[INFO] Finished Known Exploited Vulnerability Analyzer (0 seconds)
[INFO] Finished Dependency Bundling Analyzer (0 seconds)
[INFO] Finished Unused Suppression Rule Analyzer (0 seconds)
[INFO] Analysis Complete (39 seconds)
[INFO] Writing report to: /Users/rcarrata/Code/Security/spring-petclinic/./dependency-check-report.html

Dependency-check works by collecting information about the files it scans (using Analyzers). The information collected is called Evidence; there are three types of evidence collected: vendor, product, and version.

For instance, the JarAnalyzer will collect information from the Manifest, pom.xml, and the package names within the JAR files scanned. It has heuristics to place the information from various sources into one or more buckets of evidence.

If we open the report generated from the dependency-check execution, we can see a summary of the dependencies and more interesting information about CVEs, Evidence, etc:

2.3 Dependency-Check – File Type Analyzers

OWASP dependency-check contains several file type analyzers that are used to extract identification information from the files analyzed.

Because of that, it is not only analyzing Java applications or JAR artifacts; it can analyze many more file types:

2.4 Building the Container Image

Now it’s time to build our Container image!

We will use the Dockerfile provided to build the image using Podman or Docker so that we can push it to a container registry.

We can then run the image in distributed systems such as Kubernetes or OpenShift (or in other systems):

$ podman build -t sprint-petclinic:v1.0 . -f .devcontainer/Dockerfile
[+] Building 61.1s (8/8) FINISHED
 => [internal] load build definition from Dockerfile                                                                                  
=> transferring dockerfile: 627B                                                                                                  
 => [internal] load .dockerignore                                                                                                    
=> transferring context: 2B                                                                                                       
 => [internal] load metadata for mcr.microsoft.com/vscode/devcontainers/java:0-17-bullseye                                            
 => [1/4] FROM mcr.microsoft.com/vscode/devcontainers/java:0-17-bullseye@sha256:8e63b81b6dc5fa4dc9ff0bb3b707dc643e7c9cb63b70f2fe61b
=> resolve mcr.microsoft.com/vscode/devcontainers/java:0-17-bullseye@sha256:8e63b81b6dc5fa4dc9ff0bb3b707dc643e7c9cb63b70f2fe61b9
….
 => exporting to image                                                                                                                2.1s
 => exporting layers                                                                                                     
 =>writing image sha256:ca660a5bf3f86a03550121b368f312194b3a39ccc9572602c9284ff42e109ebe                                         
 =>naming to docker.io/library/sprint-petclinic:v1.0   

Now that we have the image generated, check the Container Image ID and the version:

$ podman images | grep sprint
sprint-petclinic                                              v1.0      ca660a5bf3f8   5 minutes ago   1.86GB

2.5 Syft

Syft is a CLI tool and Go library for generating a Software Bill of Materials (SBOM) from container images and filesystems.

Syft is compatible with various widely-used package formats in the most popular operating systems and programming languages. The list includes

• APK (Alpine), DEB (Debian), and RPM (Fedora) OS packages.
• Identification of Linux distributions across Alpine, CentOS, Debian, and RHEL flavors.
• Go modules
• Java in JAR, EAR, and WAR variations
• NPM and Yarn packages
• Python Wheels and Eggs
• Ruby bundles

While not all programming languages are included, you can still take advantage of the OS-level scanning regardless of the technology stack used by your application.

To generate an SBOM for a container image:

$ syft sprint-petclinic:v1.0
 ✔ Loaded image
 ✔ Parsed image
 ✔ Cataloged packages      [270 packages]

NAME                       VERSION                         TYPE
adduser                    3.118                           deb
apt                        2.2.4                           deb
apt-transport-https        2.2.4                           deb
apt-utils                  2.2.4                           deb
base-files                 11.1+deb11u6                    deb
base-passwd                3.5.51                          deb
bash                       5.1-2+deb11u1                   deb
binutils                   2.35.2-2                        deb
binutils-common            2.35.2-2                        deb
binutils-x86-64-linux-gnu  2.35.2-2                        deb
bsdextrautils              2.36.1-8+deb11u1                deb
…

The default output format is called table. It renders a columnar-based table of results in your terminal, creating a new row for each detected package:

syft sprint-petclinic:v1.0 -o json > /tmp/sprint-petclinit-sbom-v1.0.json
 ✔ Loaded image
 ✔ Parsed image
 ✔ Cataloged packages      [270 packages]

With Syft, you can extract lists of packages from your container images, which provide you with an SBOM for your image. This generated data enhances your understanding of the length of your supply chain.

We can check the output of the SBOM json file generated by Syft:

$ head /tmp/sprint-petclinit-sbom-v1.0.json
{
 “artifacts”: [
  {
   “id”: “3e9282034226b93f”,
   “name”: “adduser”,
   “version”: “3.118”,
   “type”: “deb”,
   “foundBy”: “dpkgdb-cataloger”,
   “locations”: [
    {

By incorporating Syft scans into your workflow, you will be kept updated on the packages you are utilizing. This will enable you to evaluate each package to determine its necessity. In case you come across numerous packages that are not essential for your workload, it is advisable to switch to a minimal base image and only add crucial software layers on top.

But wouldn’t it be amazing to have a user interface that could compile all dependencies, analyze SBOMs and other materials, and track and check for any CVEs that may affect our supply chain?

2.6 Dependency Track

Dependency-Track is an intelligent Component Analysis platform that allows organizations to identify and reduce risk in the software supply chain. Dependency-Track takes a unique and

highly beneficial approach by leveraging the capabilities of Software Bill of Materials (SBOM).

This approach provides capabilities that traditional Software Composition Analysis (SCA) solutions cannot achieve.

Dependency-Track monitors component usage across all versions of every application in its portfolio in order to proactively identify risk across an organization. The platform has an API-first design and is ideal for use in CI/CD environments.

After uploading the SBOM to the Dependency Track project (we created one called Secure Supply Chain Demo), we can see all the packages that are within the container image:

Alongside, we can see the Risk Score and the Vulnerabilities that affect each layer of our software, and therefore the risks that could be affecting our Supply Chain.

Furthermore, we will have a Project-Wide dashboard with the Overview of our demo Artifact and software:

3. Next Steps: Automate and Signing Images and Metadata Artifacts

Now that we have discussed the Open Source tools that we can use in our Secure Supply Chain, we need to move to the next step of the key principles of the Supply Chain Security:

• Automation: Automation is critical to supply chain security and can significantly reduce the possibility of human error and configuration drift.

• Clarity: The build environments used in a supply chain should be clearly defined, with limited scope.

Our upcoming blog post will cover the automation of all the steps discussed in this article, including container and metadata artifact signing, within our DevSecOps pipeline. Additionally, we’ll introduce other Open Source tools and projects, like Sigstore or Cosign, to further enhance the security of our Software Supply Chains.

NOTE: Opinions expressed in this blog are my own and do not necessarily reflect that of the company I work for.

Stay tuned to the next blog post!

Overview

Submariner is an open source tool that can be used with Red Hat Advanced Cluster Management for Kubernetes to provide direct networking between pods and compatible multicluster service discovery across two or more Kubernetes clusters in your environment, either on-premises or in the cloud.

Azure Red Hat OpenShift or ARO provides single-tenant, high-availability Kubernetes clusters on Azure, supported by Red Hat and Microsoft. Azure Red Hat OpenShift is jointly engineered, operated, and supported by Red Hat and Microsoft to provide an integrated support experience.

ROSA or Red Hat OpenShift on AWS is fully-managed, turnkey application platform that allows you to focus on delivering value to your customers by building and deploying applications.

Let’s discover how to set up RHACM Submariner for connecting overlay networking between ARO and ROSA clusters!

Prerequisites

To start to test this blog post, we need to have in place these prerequisites:

• OpenShift Cluster version 4 (ROSA/ARO or non-ROSA/ARO)
• az cli
• rosa cli
• aws cli (optional)

Manage Multiple Logins

• In order to manage several clusters, we will add a new Kubeconfig file to manage the logins and change quickly from one context to another:

rm -rf /var/tmp/acm-lab-kubeconfig
touch /var/tmp/acm-lab-kubeconfig
export KUBECONFIG=/var/tmp/acm-lab-kubeconfig

Deploy ACM Cluster HUB

We will use the first OpenShift cluster to deploy ACM Hub.

• Login into the HUB OpenShift cluster and set the proper context

kubectl login --username xxx --password xxx --server=https://api.cluster-xxx.xxx.xxx.xxx.com:6443

kubectl config rename-context $(oc config current-context) hub
kubectl config use hub

• Create the namespace for ACM

cat << EOF | kubectl apply -f -
apiVersion: v1
kind: Namespace
metadata:
  name: open-cluster-management
  labels:
    openshift.io/cluster-monitoring: "true"
EOF

• Create the OperatorGroup for ACM

cat << EOF | kubectl apply -f -
apiVersion: operators.coreos.com/v1
kind: OperatorGroup
metadata:
  name: open-cluster-management
  namespace: open-cluster-management
spec:
  targetNamespaces:
    - open-cluster-management
EOF

• Install Operator ACM 2.7

cat << EOF | kubectl apply -f -
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  name: advanced-cluster-management
  namespace: open-cluster-management
spec:
  channel: release-2.7
  installPlanApproval: Automatic
  name: advanced-cluster-management
  source: redhat-operators
  sourceNamespace: openshift-marketplace
EOF

NOTE: you can select ACM 2.7 onwards to install ACM Submariner for ROSA/ARO.

• Check that the Operator has installed successfully

kubectl get csv -n open-cluster-management
NAME                                 DISPLAY                                      VERSION   REPLACES                             PHASE
advanced-cluster-management.v2.7.2   Advanced Cluster Management for Kubernetes   2.7.2     advanced-cluster-management.v2.7.1   Succeeded

NOTE: ACM Submariner for ROSA clusters only works with ACM 2.7 or newer!

• Install MultiClusterHub instance in the ACM namespace

cat << EOF | kubectl apply -f -
apiVersion: operator.open-cluster-management.io/v1
kind: MultiClusterHub
metadata:
  namespace: open-cluster-management
  name: multiclusterhub
spec: {}
EOF

• Check that the MultiClusterHub is installed and running properly

kubectl get multiclusterhub -n open-cluster-management -o json | jq '.items[0].status.phase'
"Running"

NOTE: if it’s not in Running state, wait a couple of minutes and check again.

Deploy ROSA Cluster

• Define the prerequisites to install the ROSA cluster

 export VERSION=4.11.36 \
        ROSA_CLUSTER_NAME=rosa-sbmr1 \
        AWS_ACCOUNT_ID=`aws sts get-caller-identity --query Account --output text` \
        REGION=eu-west-1 \
        AWS_PAGER="" \
        CIDR="10.10.0.0/16"

NOTE: it’s critical that the Machine CIDRs of the ROSA and ARO clusters do not overlap. For that reason, we’re setting different CIDRs than the out-of-the-box ROSA / ARO cluster install.

• Create the IAM Account Roles

rosa create account-roles --mode auto --yes

• Generate a STS ROSA cluster

rosa create cluster -y --cluster-name ${ROSA_CLUSTER_NAME} \
--region ${REGION} --version ${VERSION} \
--machine-cidr $CIDR \
--sts

• Create the Operator and OIDC Roles

rosa create operator-roles --cluster ${ROSA_CLUSTER_NAME} --mode auto --yes
rosa create oidc-provider --cluster ${ROSA_CLUSTER_NAME} --mode auto --yes

• Check the status of the Rosa cluster (40 mins wait until is in ready status)

rosa describe cluster --cluster ${ROSA_CLUSTER_NAME} | grep State
State:                      ready

• Set the admin user for the ROSA cluster

rosa create admin --cluster=$ROSA_CLUSTER_NAME

• Login into the rosa cluster and set the proper context

oc login https://api.rosa-sbmr1.xxx.xxx.xxx.com:6443 --username cluster-admin --password xxx

kubectl config rename-context $(oc config current-context) $ROSA_CLUSTER_NAME
kubectl config use $ROSA_CLUSTER_NAME

kubectl get dns cluster -o jsonpath='{.spec.baseDomain}'

Generate ROSA New nodes for submariner

• Create new node/s that will be used to run Submariner gateway using the following command (check https://github.com/submariner-io/submariner/issues/1896 for more details)

rosa create machinepool --cluster $ROSA_CLUSTER_NAME --name=sm-gw-mp --replicas=1 --labels='submariner.io/gateway=true'

NOTE: setting replicas=2 means that we allocate two nodes for SM GW , to support GW Active/Passive HA (check Gateway Failover section ), if GW HA is not needed you can set replicas=1.

• Check the machinepools requested, including the submariner machinepool requested

rosa list machinepools -c $ROSA_CLUSTER_NAME
ID        AUTOSCALING  REPLICAS  INSTANCE TYPE  LABELS                        TAINTS    AVAILABILITY ZONES    SPOT INSTANCES
Default   No           2         m5.xlarge                                              eu-west-1a            N/A
sm-gw-mp  No           1         m5.xlarge      submariner.io/gateway=true              eu-west-1a            No

• After a couple of minutes, check the new nodes generated

kubectl get nodes --show-labels | grep submariner

Deploy ARO Cluster

IMPORTANT: To enable Submariner in ROSA - ARO clusters, the POD_CIDR and SERVICE_CIDR can’t overlap between them. To avoid IP address conflicts, the ARO cluster needs to modify the default IP CIDRs. Check the Submariner docs for more information.

• Define the prerequisites to install the ARO cluster

AZR_RESOURCE_LOCATION=eastus
AZR_RESOURCE_GROUP=aro-sbmr2-rg
AZR_CLUSTER=aro-sbmr2
AZR_PULL_SECRET=~/Downloads/pull-secret.txt
POD_CIDR="10.132.0.0/14"
SERVICE_CIDR="172.31.0.0/16"

• Create an Azure resource group

 az group create \
   --name $AZR_RESOURCE_GROUP \
   --location $AZR_RESOURCE_LOCATION

• Create virtual network

 az network vnet create \
   --address-prefixes 10.0.0.0/22 \
   --name "$AZR_CLUSTER-aro-vnet-$AZR_RESOURCE_LOCATION" \
   --resource-group $AZR_RESOURCE_GROUP

• Create control plane subnet

 az network vnet subnet create \
   --resource-group $AZR_RESOURCE_GROUP \
   --vnet-name "$AZR_CLUSTER-aro-vnet-$AZR_RESOURCE_LOCATION" \
   --name "$AZR_CLUSTER-aro-control-subnet-$AZR_RESOURCE_LOCATION" \
   --address-prefixes 10.0.0.0/23 \
   --service-endpoints Microsoft.ContainerRegistry

• Create machine subnet

az network vnet subnet create \
  --resource-group $AZR_RESOURCE_GROUP \
  --vnet-name "$AZR_CLUSTER-aro-vnet-$AZR_RESOURCE_LOCATION" \
  --name "$AZR_CLUSTER-aro-machine-subnet-$AZR_RESOURCE_LOCATION" \
  --address-prefixes 10.0.2.0/23 \
  --service-endpoints Microsoft.ContainerRegistry

• Disable network policies on the control plane subnet

az network vnet subnet update \
  --name "$AZR_CLUSTER-aro-control-subnet-$AZR_RESOURCE_LOCATION" \
  --resource-group $AZR_RESOURCE_GROUP \
  --vnet-name "$AZR_CLUSTER-aro-vnet-$AZR_RESOURCE_LOCATION" \
  --disable-private-link-service-network-policies true

• Create the ARO cluster

 az aro create \
   --resource-group $AZR_RESOURCE_GROUP \
   --name $AZR_CLUSTER \
   --vnet "$AZR_CLUSTER-aro-vnet-$AZR_RESOURCE_LOCATION" \
   --master-subnet "$AZR_CLUSTER-aro-control-subnet-$AZR_RESOURCE_LOCATION" \
   --worker-subnet "$AZR_CLUSTER-aro-machine-subnet-$AZR_RESOURCE_LOCATION" \
   --pod-cidr "$POD_CIDR" \
   --service-cidr "$SERVICE_CIDR" \
   --pull-secret @$AZR_PULL_SECRET

• Get ARO OpenShift API Url

ARO_URL=$(az aro show -g $AZR_RESOURCE_GROUP -n $AZR_CLUSTER --query apiserverProfile.url -o tsv)

• Login into the ARO cluster and set context

ARO_KUBEPASS=$(az aro list-credentials --name $AZR_CLUSTER --resource-group $AZR_RESOURCE_GROUP -o tsv --query kubeadminPassword)

• Login into the ARO cluster and set context

kubectl login --username kubeadmin --password $ARO_KUBEPASS --server=$ARO_URL

kubectl config rename-context $(oc config current-context) $AZR_CLUSTER
kubectl config use $AZR_CLUSTER

kubectl get dns cluster -o jsonpath='{.spec.baseDomain}'

NOTE: ARO doesn’t need extra nodes to have the ACM Submariner components deployed.

Create ManagedClusterSets

• Create a ManagedClusterSet for ROSA and ARO clusters

kubectl config use hub
kubectl get dns cluster -o jsonpath='{.spec.baseDomain}'

cat << EOF | kubectl apply -f -
apiVersion: cluster.open-cluster-management.io/v1beta1
kind: ManagedClusterSet
metadata:
  name: rosa-aro-clusters
EOF

Import ROSA cluster in ACM (CLI)

We will import the cluster using the auto-import secret and using the Klusterlet Addon Config.

If you want to import your cluster using the RHACM UI, refer to the official Importing a managed cluster by using console documentation.

• Retrieve ROSA TOKEN the ROSA API from the ROSA cluster

kubectl config use $ROSA_CLUSTER_NAME
SUB1_API=$(oc whoami --show-server)
echo "$ROSA_CLUSTER_NAME API: $SUB1_API\n"

SUB1_TOKEN=$(oc whoami -t)
echo "$ROSA_CLUSTER_NAME Token: $SUB1_TOKEN\n"

• Config the Hub as the current context

kubectl config use hub
kubectl get dns cluster -o jsonpath='{.spec.baseDomain}'

• Create (in ACM Hub cluster) ManagedCluster object defining the ROSA cluster

cat << EOF | kubectl apply -f -
apiVersion: cluster.open-cluster-management.io/v1
kind: ManagedCluster
metadata:
  name: $ROSA_CLUSTER_NAME
  labels:
    name: $ROSA_CLUSTER_NAME
    cloud: auto-detect
    vendor: auto-detect
    cluster.open-cluster-management.io/clusterset: rosa-aro-clusters
    env: $ROSA_CLUSTER_NAME
  annotations: {}
spec:
  hubAcceptsClient: true
EOF

• Create (in ACM Hub cluster) auto-import-secret.yaml secret defining the token and server from the ROSA cluster:

cat << EOF | kubectl apply -f -
apiVersion: v1
kind: Secret
metadata:
  name: auto-import-secret
  namespace: $ROSA_CLUSTER_NAME
stringData:
  autoImportRetry: "2"
  token: "${SUB1_TOKEN}"
  server: "${SUB1_API}"
type: Opaque
EOF

• Create and apply the klusterlet add-on configuration file for the ROSA cluster

cat << EOF | kubectl apply -f -
apiVersion: agent.open-cluster-management.io/v1
kind: KlusterletAddonConfig
metadata:
  name: $ROSA_CLUSTER_NAME
  namespace: $ROSA_CLUSTER_NAME
spec:
  clusterName: $ROSA_CLUSTER_NAME
  clusterNamespace: $ROSA_CLUSTER_NAME
  clusterLabels:
    name: $ROSA_CLUSTER_NAME
    cloud: auto-detect
    vendor: auto-detect
    cluster.open-cluster-management.io/clusterset: rosa-aro-clusters
    env: $ROSA_CLUSTER_NAME
  applicationManager:
    enabled: true
  policyController:
    enabled: true
  searchCollector:
    enabled: true
  certPolicyController:
    enabled: true
  iamPolicyController:
    enabled: true
EOF

• Check the imported cluster in ACM

kubectl get ManagedCluster
NAME            HUB ACCEPTED   MANAGED CLUSTER URLS                                           JOINED   AVAILABLE   AGE
local-cluster   true           https://api.cluster-xxxx.xxxx.xxxx.xxx.com:6443   True     True        5h9m
rosa-sbmr1      true           https://api.rosa-subm1.xxxx.p1.openshiftapps.com:6443          True     True        1m

Import ARO cluster into ACM (CLI)

• Retrieve the ARO token and the ARO API URL from the ARO cluster

kubectl config use $AZR_CLUSTER

SUB2_API=$(oc whoami --show-server)
echo "$AZR_CLUSTER API: $SUB2_API\n"

SUB2_TOKEN=$(oc whoami -t)
echo "$AZR_CLUSTER Token: $SUB2_TOKEN\n"

• Config the Hub as the current context

kubectl config use hub
kubectl get mch -A

• Create (in the Hub) ManagedCluster object defining the ARO cluster:

cat << EOF | kubectl apply -f -
apiVersion: cluster.open-cluster-management.io/v1
kind: ManagedCluster
metadata:
  name: $AZR_CLUSTER
  labels:
    name: $AZR_CLUSTER
    cloud: auto-detect
    vendor: auto-detect
    cluster.open-cluster-management.io/clusterset: rosa-aro-clusters
    env: $AZR_CLUSTER
  annotations: {}
spec:
  hubAcceptsClient: true
EOF

• Create (in the Hub) auto-import-secret.yaml secret defining the token and server from the ARO cluster:

cat << EOF | kubectl apply -f -
apiVersion: v1
kind: Secret
metadata:
  name: auto-import-secret
  namespace: $AZR_CLUSTER
stringData:
  autoImportRetry: "2"
  token: "${SUB2_TOKEN}"
  server: "${SUB2_API}"
type: Opaque
EOF

cat << EOF | kubectl apply -f -
apiVersion: agent.open-cluster-management.io/v1
kind: KlusterletAddonConfig
metadata:
  name: $AZR_CLUSTER
  namespace: $AZR_CLUSTER
spec:
  clusterName: $AZR_CLUSTER
  clusterNamespace: $AZR_CLUSTER
  clusterLabels:
    Name: $AZR_CLUSTER
    cloud: auto-detect
    vendor: auto-detect
    cluster.open-cluster-management.io/clusterset: rosa-aro-clusters
    env: $AZR_CLUSTER
  applicationManager:
    enabled: true
  policyController:
    enabled: true
  searchCollector:
    enabled: true
  certPolicyController:
    enabled: true
  iamPolicyController:
    enabled: true
EOF

Review the clusters imported in ACM

• Check the managed clusters in ACM

kubectl config use hub

kubectl get managedclusters
NAME            HUB ACCEPTED   MANAGED CLUSTER URLS                                           JOINED   AVAILABLE   AGE
aro-submr2      true           https://api.xxxx.xxxx.xxxx:6443                             True     True        2m34s
local-cluster   true           https://api.cluster-xxxx.xxxx.xxxx.xxxx.com:6443   True     True        2d
rosa-sbmr1      true           https://api.rosa-xxxx.xxxx.p1.openshiftapps.com:6443          True     True        46h

Now it’s time to deploy submariner in our Managed Clusters (ROSA and ARO). Either deploy using the RHACM UI or with CLI (choose one).

Deploy Submariner Addon in Managed ROSA and ARO clusters from the RHACM UI

• Inside the ClusterSets tab, go to the rosa-aro-clusters generated.

• Go to Submariner add-ons and Click in “Install Submariner Add-Ons”

• Configure the Submariner addons adding both ROSA and ARO clusters generated:

The Submariner Add-on installation will start, and will take up to 10 minutes to finish.

Deploy Submariner Addon in Managed ROSA and ARO clusters with CLI

NOTE: All of these commands are executed in the ACM Hub cluster, not in the ACM Managed Clusters (ROSA / ARO created).

• After the ManagedClusterSet is created, the submariner-addon creates a namespace called managed-cluster-set-name-broker and deploys the Submariner broker to it.

kubectl get ns | grep broker
default-broker                                     Active   2d
rosa-aro-clusters-broker                           Active   8m1s

• Create the Broker configuration on the hub cluster in the rosa-clusters-broker namespace:

cat << EOF | kubectl apply -f -
apiVersion: submariner.io/v1alpha1
kind: Broker
metadata:
     name: submariner-broker
     namespace: rosa-aro-clusters-broker
spec:
     globalnetEnabled: false
EOF

NOTE: Set the value of globalnetEnabled to true if you want to enable Submariner Globalnet in the ManagedClusterSet.

• Check the Submariner Broker in the rosa-clusters-broker namespace:

$ kubectl get broker -n rosa-aro-clusters-broker
NAME                AGE
submariner-broker   5s

• Deploy the SubmarinerConfig for the ROSA cluster imported:

cat << EOF | kubectl apply -f -
apiVersion: submarineraddon.open-cluster-management.io/v1alpha1
kind: SubmarinerConfig
metadata:
  name: submariner
  namespace: $ROSA_CLUSTER_NAME
spec:
  IPSecNATTPort: 4500
  NATTEnable: true
  cableDriver: libreswan
  loadBalancerEnable: true
EOF

• Deploy the SubmarinerConfig for the ARO cluster imported:

cat << EOF | kubectl apply -f -
apiVersion: submarineraddon.open-cluster-management.io/v1alpha1
kind: SubmarinerConfig
metadata:
  name: submariner
  namespace: $AZR_CLUSTER
spec:
  IPSecNATTPort: 4500
  NATTEnable: true
  cableDriver: libreswan
  loadBalancerEnable: true
EOF

• Deploy Submariner on the ROSA cluster:

cat << EOF | kubectl apply -f -
apiVersion: addon.open-cluster-management.io/v1alpha1
kind: ManagedClusterAddOn
metadata:
     name: submariner
     namespace: $ROSA_CLUSTER_NAME
spec:
     installNamespace: submariner-operator
EOF

• Deploy Submariner on the ARO cluster:

cat << EOF | kubectl apply -f -
apiVersion: addon.open-cluster-management.io/v1alpha1
kind: ManagedClusterAddOn
metadata:
     name: submariner
     namespace: $AZR_CLUSTER
spec:
     installNamespace: submariner-operator
EOF

The Submariner Add-on installation will start, and will take up to 10 minutes to finish.

Check the Status of the Submariner Networking Add-On

• A few minutes later (up to 10 minutes), we can check that the app Connection Status and the Agent Status are Healthy:

NOTE: Opinions expressed in this blog are my own and do not necessarily reflect that of the company I work for.

Happy submarining!

Overview

Submariner is an open source tool that can be used with Red Hat Advanced Cluster Management for Kubernetes to provide direct networking between pods and compatible multicluster service discovery across two or more Kubernetes clusters in your environment, either on-premises or in the cloud.

ROSA or Red Hat OpenShift on AWS is fully-managed, turnkey application platform that allows you to focus on delivering value to your customers by building and deploying applications.

For this blog post there are some prerequisites that need to be in place:

• OpenShift Cluster version 4 (ROSA or non-ROSA)
• ROSA cli
• AWS cli (optional)
• ACM 2.7 or newer

NOTE: ACM Submariner for ROSA clusters only works with ACM 2.7 or newer!

Manage Multiple Logins

• In order to manage several clusters, we will add a new Kubeconfig file to manage the logins and change quickly from one context to another:

rm -rf /var/tmp/acm-lab-kubeconfig
touch /var/tmp/acm-lab-kubeconfig
export KUBECONFIG=/var/tmp/acm-lab-kubeconfig

Deploy ACM Cluster HUB

We will use the first OpenShift cluster to deploy ACM Hub.

• Login into the HUB OpenShift cluster and set the proper context:

oc login --username xxx --password xxx --server=https://api.cluster-xxx.xxx.xxx.xxx.com:6443

kubectl config rename-context $(oc config current-context) hub
kubectl config use hub

• Create the namespace for ACM

cat << EOF | kubectl apply -f -
apiVersion: v1
kind: Namespace
metadata:
  name: open-cluster-management
  labels:
    openshift.io/cluster-monitoring: "true"
EOF

• Create the OperatorGroup for ACM

cat << EOF | kubectl apply -f -
apiVersion: operators.coreos.com/v1
kind: OperatorGroup
metadata:
  name: open-cluster-management
  namespace: open-cluster-management
spec:
  targetNamespaces:
    - open-cluster-management
EOF

• Install Operator ACM 2.7

cat << EOF | kubectl apply -f -
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  name: advanced-cluster-management
  namespace: open-cluster-management
spec:
  channel: release-2.7
  installPlanApproval: Automatic
  name: advanced-cluster-management
  source: redhat-operators
  sourceNamespace: openshift-marketplace
EOF

• Check that the Operator has installed successfully

oc get csv
NAME                                 DISPLAY                                      VERSION   REPLACES   PHASE
advanced-cluster-management.v2.7.0   Advanced Cluster Management for Kubernetes   2.7.0                Succeeded

NOTE: ACM Submariner will only work from 2.7 onwards! Ensure that you have a >= 2.7 ACM version.

• Install MultiClusterHub instance in the ACM namespace

cat << EOF | kubectl apply -f -
apiVersion: operator.open-cluster-management.io/v1
kind: MultiClusterHub
metadata:
  namespace: open-cluster-management
  name: multiclusterhub
spec: {}
EOF

• Check that the MultiClusterHub is properly installed

kubectl get mch -n open-cluster-management multiclusterhub -o jsonpath='{.status.phase}'

NOTE: if it’s not in Running state, wait a couple of minutes and check again.

Deploy First ROSA Cluster

• Define the prerequisites to install the ROSA cluster

 export VERSION=4.11.36 \
        ROSA_CLUSTER_NAME_1=rosa-sbmr1 \
        AWS_ACCOUNT_ID=`aws sts get-caller-identity --query Account --output text` \
        REGION=eu-west-1 \
        AWS_PAGER="" \
        CIDR="10.0.0.0/16"

NOTE: it’s critical that the Machine CIDRs of the ROSA clusters do not overlap. For that reason, we’re setting different CIDRs than the out-of-the-box ROSA cluster install.

• Create the IAM Account Roles

rosa create account-roles --mode auto --yes

• Generate a STS ROSA cluster

rosa create cluster -y --cluster-name ${ROSA_CLUSTER_NAME_1} \
--region ${REGION} --version ${VERSION} \
--machine-cidr $CIDR \
--sts

• Create the Operator and OIDC Roles

rosa create operator-roles --cluster ${ROSA_CLUSTER_NAME_1} --mode auto --yes
rosa create oidc-provider --cluster ${ROSA_CLUSTER_NAME_1} --mode auto --yes

• Check the status of the Rosa cluster (40 mins wait until is in ready status)

rosa describe cluster --cluster ${ROSA_CLUSTER_NAME_1} | grep State
State:                      ready

• Set the admin user for the ROSA cluster

rosa create admin --cluster=$ROSA_CLUSTER_NAME_1

• Login into the rosa cluster and set the proper context

oc login https://api.rosa-sbmr1.xxx.xxx.xxx.com:6443 --username cluster-admin --password xxx

kubectl config rename-context $(oc config current-context) $ROSA_CLUSTER_NAME_1
kubectl config use $ROSA_CLUSTER_NAME_1

kubectl get dns cluster -o jsonpath='{.spec.baseDomain}'

Generate ROSA New nodes for submariner

• Create new node/s that will be used to run Submariner gateway using the following command (check the related GitHub issue for more details)

rosa create machinepool --cluster $ROSA_CLUSTER_NAME_1 --name=sm-gw-mp --replicas=1 --labels='submariner.io/gateway=true'

NOTE: setting replicas=2 means that we allocate two nodes for SM GW , to support GW Active/Passive HA (check Gateway Failover section ), if GW HA is not needed you can set replicas=1.

• Check the machinepools requested, including the submariner machinepool requested

rosa list machinepools -c $ROSA_CLUSTER_NAME_1
ID        AUTOSCALING  REPLICAS  INSTANCE TYPE  LABELS                        TAINTS    AVAILABILITY ZONES    SPOT INSTANCES
Default   No           2         m5.xlarge                                              eu-west-1a            N/A
sm-gw-mp  No           2         m5.xlarge      submariner.io/gateway=true              eu-west-1a            No

• After a couple of minutes, check the new nodes generated

kubectl get nodes --show-labels | grep submariner

Deploy Second ROSA Cluster

IMPORTANT: To enable Submariner in both ROSA clusters, the POD_CIDR and SERVICE_CIDR can’t overlap between them. To avoid IP address conflicts, the second ROSA cluster needs to modify the default IP CIDRs. Check the Submariner docs for more information.

• Define the prerequisites to install the second ROSA cluster

 export VERSION=4.11.36 \
        ROSA_CLUSTER_NAME_2=rosa-sbmr2 \
        AWS_ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text) \
        REGION=us-east-2 \
        AWS_PAGER="" \
        CIDR="10.20.0.0/16" \
        POD_CIDR="10.132.0.0/14" \
        SERVICE_CIDR="172.31.0.0/16"

• Create the IAM Account Roles

rosa create account-roles --mode auto --yes

• Generate the second STS ROSA cluster (with the POD_CIDR and SERVICE_CIDR modified)

 rosa create cluster -y --cluster-name ${ROSA_CLUSTER_NAME_2} \
   --region ${REGION} --version ${VERSION} \
   --machine-cidr $CIDR \
   --pod-cidr $POD_CIDR \
   --service-cidr $SERVICE_CIDR \
   --sts

• Create the Operator and OIDC Roles

rosa create operator-roles -c $ROSA_CLUSTER_NAME_2 --mode auto --yes
rosa create oidc-provider -c $ROSA_CLUSTER_NAME_2 --mode auto --yes

• Check the status of the Rosa cluster (40 mins wait until is in ready status)

rosa describe cluster --cluster ${ROSA_CLUSTER_NAME_2} | grep State
State:                      ready

• Set the admin user for the ROSA cluster

rosa create admin --cluster=$ROSA_CLUSTER_NAME_2

• Login into the rosa cluster and set the proper context

oc login https://api.rosa-sbmr2.xxx.xxx.xxx.com:6443 --username cluster-admin --password xxx


kubectl config rename-context $(oc config current-context) $ROSA_CLUSTER_NAME_2
kubectl config use $ROSA_CLUSTER_NAME_2

kubectl get dns cluster -o jsonpath='{.spec.baseDomain}'

Generate ROSA New nodes for submariner

• Create new node/s that will be used to run Submariner gateway using the following command

rosa create machinepool --cluster $ROSA_CLUSTER_NAME_2 --name=sm-gw-mp --replicas=1 --labels='submariner.io/gateway=true'

• Check the machinepools requested, including the submariner machinepool requested:

rosa list machinepools -c $ROSA_CLUSTER_NAME_2
ID        AUTOSCALING  REPLICAS  INSTANCE TYPE  LABELS                        TAINTS    AVAILABILITY ZONES    SPOT INSTANCES
Default   No           2         m5.xlarge                                              us-east-2a            N/A
sm-gw-mp  No           2         m5.xlarge      submariner.io/gateway=true              us-east-2a            No

• After a couple of minutes, check the new nodes generated

kubectl get nodes --show-labels | grep submariner

Create a ManagedClusterSet

• In the Hub (where ACM is installed), create the ManagedClusterSet for the rosa-clusters:

kubectl config use hub
kubectl get mch -A

cat << EOF | kubectl apply -f -
apiVersion: cluster.open-cluster-management.io/v1beta1
kind: ManagedClusterSet
metadata:
  name: rosa-clusters
EOF

Import ROSA Sub1

We will import the cluster using the auto-import secret and using the Klusterlet Addon Config.

• Retrieve ROSA TOKEN the ROSA API from the first ROSA cluster

kubectl config use $ROSA_CLUSTER_NAME_1
SUB1_TOKEN=$(oc whoami -t)
echo $SUB1_TOKEN
SUB1_API=$(oc whoami --show-server)
echo $SUB1_API

• Config the Hub as the current context

kubectl config use hub
kubectl get dns cluster -o jsonpath='{.spec.baseDomain}'
kubectl get mch -A

• Create (in the Hub) ManagedCluster object defining the rosa-subm1 cluster

cat << EOF | kubectl apply -f -
apiVersion: cluster.open-cluster-management.io/v1
kind: ManagedCluster
metadata:
  name: $ROSA_CLUSTER_NAME_1
  labels:
    name: $ROSA_CLUSTER_NAME_1
    cluster.open-cluster-management.io/clusterset: rosa-clusters
  annotations: {}
spec:
  hubAcceptsClient: true
EOF

• Create (in the Hub) auto-import-secret.yaml secret defining the token and server from the first ROSA cluster

cat << EOF | kubectl apply -f -
apiVersion: v1
kind: Secret
metadata:
  name: auto-import-secret
  namespace: $ROSA_CLUSTER_NAME_1
stringData:
  autoImportRetry: "5"
  token: ${SUB1_TOKEN}
  server: ${SUB1_API}
type: Opaque
EOF

• Create and apply the klusterlet add-on configuration file for the first rosa cluster

cat << EOF | kubectl apply -f -
apiVersion: agent.open-cluster-management.io/v1
kind: KlusterletAddonConfig
metadata:
  name: $ROSA_CLUSTER_NAME_1
  namespace: $ROSA_CLUSTER_NAME_1
spec:
  clusterName: $ROSA_CLUSTER_NAME_1
  clusterNamespace: $ROSA_CLUSTER_NAME_1
  clusterLabels:
    name: $ROSA_CLUSTER_NAME_1
    cloud: auto-detect
    vendor: auto-detect
    cluster.open-cluster-management.io/clusterset: rosa-clusters
  applicationManager:
    enabled: true
  certPolicyController:
    enabled: true
  iamPolicyController:
    enabled: true
  policyController:
    enabled: true
  searchCollector:
    enabled: true
EOF

Import ROSA sub2 (CLI)

• Retrieve ROSA TOKEN the ROSA API from the second ROSA cluster

kubectl config use $ROSA_CLUSTER_NAME_2
SUB2_API=$(oc whoami --show-server)
echo "$ROSA_CLUSTER_NAME_2 API: $SUB2_API\n"

SUB2_TOKEN=$(oc whoami -t)
echo "$ROSA_CLUSTER_NAME_2 Token: $SUB2_TOKEN\n"

• Config the Hub as the current context

kubectl config use hub
kubectl get mch -A

• Create (in the Hub) ManagedCluster object defining the second ROSA cluster

cat << EOF | kubectl apply -f -
apiVersion: cluster.open-cluster-management.io/v1
kind: ManagedCluster
metadata:
  name: $ROSA_CLUSTER_NAME_2
  labels:
    name: $ROSA_CLUSTER_NAME_2
    cloud: auto-detect
    vendor: auto-detect
    cluster.open-cluster-management.io/clusterset: rosa-clusters
    env: $ROSA_CLUSTER_NAME_2
  annotations: {}
spec:
  hubAcceptsClient: true
EOF

• Create (in the Hub) auto-import-secret.yaml secret defining the token and server from the second ROSA cluster

cat << EOF | kubectl apply -f -
apiVersion: v1
kind: Secret
metadata:
  name: auto-import-secret
  namespace: $ROSA_CLUSTER_NAME_2
stringData:
  autoImportRetry: "2"
  token: "${SUB2_TOKEN}"
  server: "${SUB2_API}"
type: Opaque
EOF

• Create and apply the klusterlet add-on configuration file for the second rosa cluster

cat << EOF | kubectl apply -f -
apiVersion: agent.open-cluster-management.io/v1
kind: KlusterletAddonConfig
metadata:
  name: $ROSA_CLUSTER_NAME_2
  namespace: $ROSA_CLUSTER_NAME_2
spec:
  clusterName: $ROSA_CLUSTER_NAME_2
  clusterNamespace: $ROSA_CLUSTER_NAME_2
  clusterLabels:
    name: $ROSA_CLUSTER_NAME_2
    cloud: auto-detect
    vendor: auto-detect
    cluster.open-cluster-management.io/clusterset: rosa-clusters
    env: rosa-subm2
  applicationManager:
    enabled: true
  policyController:
    enabled: true
  searchCollector:
    enabled: true
  certPolicyController:
    enabled: true
  iamPolicyController:
    enabled: true
EOF

• Check the managed clusters and the managed cluster set

kubectl config use hub

kubectl get managedclusters
NAME            HUB ACCEPTED   MANAGED CLUSTER URLS                                           JOINED   AVAILABLE   AGE
local-cluster   true           https://api.cluster-xxx.xxx.xxx.xxx.com:6443   True     True        5h55m
rosa-subm1      true           https://api.rosa-subm1.xxx.p1.openshiftapps.com:6443          True     True        133m
rosa-subm2      true           https://api.rosa-subm2.xxx.p1.openshiftapps.com:6443          True     True        51m

Now it’s time to deploy submariner in our Managed ROSA Clusters. Either deploy using the RHACM UI or with CLI (choose one).

Deploy Submariner Addon in Managed ROSA clusters from the RHACM UI

• Inside the ClusterSets tab, go to the rosa-aro-clusters generated.

• Go to Submariner add-ons and Click in “Install Submariner Add-Ons”

• Configure the Submariner addons adding both ROSA clusters generated:

Deploy Submariner Addon in ROSA clusters

• After the ManagedClusterSet is created, the submariner-addon creates a namespace called managed-cluster-set-name-broker and deploys the Submariner broker to it.

$ kubectl get ns | grep broker
default-broker                                     Active   6h39m
rosa-clusters-broker                               Active   13m

• Create the Broker configuration on the hub cluster in the managed-cluster-set-name-broker namespace

cat << EOF | kubectl apply -f -
apiVersion: submariner.io/v1alpha1
kind: Broker
metadata:
     name: submariner-broker
     namespace: rosa-clusters-broker
spec:
     globalnetEnabled: false
EOF

NOTE: Set the value of globalnetEnabled: true if you want to enable Submariner Globalnet in the ManagedClusterSet.

• Check the Submariner Broker in the rosa-clusters-broker namespace:

kubectl get broker -n rosa-clusters-broker
NAME                AGE
submariner-broker   21s

• We don’t need to label the ManagedCluster because it was imported with the proper labels within the proper ManagedClusterSet.

• Deploy SubmarinerConfig for the first rosa cluster imported:

cat << EOF | kubectl apply -f -
apiVersion: submarineraddon.open-cluster-management.io/v1alpha1
kind: SubmarinerConfig
metadata:
  name: submariner
  namespace: $ROSA_CLUSTER_NAME_1
spec:
  IPSecNATTPort: 4500
  NATTEnable: true
  cableDriver: libreswan
  loadBalancerEnable: true
EOF

• Deploy SubmarinerConfig for the second rosa cluster imported:

cat << EOF | kubectl apply -f -
apiVersion: submarineraddon.open-cluster-management.io/v1alpha1
kind: SubmarinerConfig
metadata:
  name: submariner
  namespace: $ROSA_CLUSTER_NAME_2
spec:
  IPSecNATTPort: 4500
  NATTEnable: true
  cableDriver: libreswan
  loadBalancerEnable: true
EOF

• Deploy Submariner on the first ROSA cluster:

cat << EOF | kubectl apply -f -
apiVersion: addon.open-cluster-management.io/v1alpha1
kind: ManagedClusterAddOn
metadata:
     name: submariner
     namespace: $ROSA_CLUSTER_NAME_1
spec:
     installNamespace: submariner-operator
EOF

• Deploy Submariner on the second ROSA cluster:

cat << EOF | kubectl apply -f -
apiVersion: addon.open-cluster-management.io/v1alpha1
kind: ManagedClusterAddOn
metadata:
     name: submariner
     namespace: $ROSA_CLUSTER_NAME_2
spec:
     installNamespace: submariner-operator
EOF

• Check the submariner status of managedclusteraddons to verify that submariner is deployed correctly

kubectl get managedclusteraddon -A | grep submariner
rosa-sbmr1      submariner                    True
rosa-sbmr2      submariner                    True

The Submariner Add-on installation will start, and will take up to 10 minutes to finish.

Check the Status of the Submariner Networking Add-On

A few minutes (up to 10 minutes) after we can check that the app Connection Status and the Agent Status are Healthy:

Testing Submariner Networking connectivity with an example app (Optional)

This final step (totally optional), is an extra step to check if the Submariner networking tunnels are built and connected properly.

This example app deploys one FE (guestbook) in the first ROSA cluster, and two Redis instances with active-backup replication.

One Redis will be in the first ROSA cluster and will sync and replicate the data inserted by the FE to the second Redis (in backup/passive mode) using the Submariner tunnel (connecting both ROSA clusters).

The connection will use the ServiceExport feature (DNS Discovery) from Submariner, which allows calling the Redis Service (Active or Passive) from within the Service CIDR.

• Clone the example repo app

git clone https://github.com/rh-mobb/acm-demo-app

• Deploy the GuestBook App in ROSA Cluster 1

kubectl config use hub
oc apply -k guestbook-app/acm-resources

• Deploy the Redis Master App in ROSA Cluster 1

oc apply -k redis-master-app/acm-resources

• Apply relaxed scc only for this PoC

kubectl config use $ROSA_CLUSTER_NAME
oc adm policy add-scc-to-user anyuid -z default -n guestbook
oc delete pod --all -n guestbook

• Deploy the Redis Slave App in ROSA Cluster 2

kubectl config use hub
oc apply -k redis-slave-app/acm-resources

• Apply relaxed SCC only for this PoC

kubectl config use $ROSA_CLUSTER_NAME_2
oc adm policy add-scc-to-user anyuid -z default -n guestbook
oc delete pod --all -n guestbook

Testing the Synchronization of the Redis Master-Slave between clusters and interacting with our FrontEnd using Submariner tunnels

To test the sync between the data from the Redis Master<->Slave, let’s write some data into our frontend. Access the route of the guestbook App and write some data:

• Now let’s see the logs in the Redis Slave:

The sync is automatic and almost instantaneous between Master-Slave.

• We can check the data written in the redis-slave with the redis-cli and the following command:

for key in $(redis-cli -p 6379 keys \*);
  do echo "Key : '$key'"
     redis-cli -p 6379 GET $key;
done

• Let’s do this in the redis-slave pod:

NOTE: Opinions expressed in this blog are my own and do not necessarily reflect that of the company I work for.

And that’s how the Redis-Master in ROSA cluster 1 properly syncs the data to the redis-slave in ROSA Cluster 2, using Submariner tunnels, all encrypted with IPSec.

Happy submarining!