BDB On-Cloud

In modern cloud infrastructure architecture, deploying a BDB platform involves a strategic amalgamation of various services and components to ensure scalability, security, and efficiency.

BDB platform setup includes managed Kubernetes services, storage as a file system, Web Application Firewall (WAF) to deter unwanted access, Load Balancer (LB), container registry, and a virtual network with deployment in a private subnet.
By meticulously orchestrating these elements, organizations can construct a robust, scalable, and secure cloud-native environment to host BDB applications.

;

Services Overview

In deploying the BDB platform, several key services are utilized:

Managed Kubernetes Service

This service offers a container orchestration platform that simplifies deployment, management, and scaling of containerized applications.

Storage as a File System

This service provides persistent storage for applications running in Kubernetes clusters, ensuring data persistence even across container restarts or rescheduling.

WAF for Security

The Web Application Firewall (WAF) protects applications from various cyber threats by inspecting HTTP traffic and enforcing security policies to block malicious requests.

Load Balancer (LB)

Load balancers distribute incoming network traffic across multiple backend servers or nodes to enhance scalability, optimize resource utilization, and improve application availability.

Container Registry

Container registries serve as centralized repositories for storing, managing, and distributing container images, facilitating efficient deployment and collaboration among development teams.

Virtual Network with Deployment in Private Subnet

Resources are deployed within a virtual network with a private subnet, enhancing security by isolating them from the public internet and other external networks.

Implementation Considerations

Namespace Management

Utilize Kubernetes namespaces to create logical boundaries between different environments and tenants. Each environment (Dev, Stg, Prod) and tenant should have its own dedicated namespace to isolate resources and configurations.

Resource Quotas and Limits

Enforce resource quotas and limits to prevent resource contention and ensure fair allocation of resources among different environments and tenants. Set limits on CPU, memory, and storage to prevent one tenant or environment from monopolizing resources.

Monitoring and Logging

Deploy monitoring and logging solutions to monitor the health, performance, and security of the platform infrastructure and applications. Utilize tools such as Prometheus, Grafana, and Elasticsearch to collect, analyze, and visualize metrics and logs from different environments and tenants.

Continuous Integration and Deployment (CI/CD)

Implement CI/CD pipelines to automate the build, test, and deployment processes across Dev, Stg, and Prod environments. Leverage tools like GitHub, GitLab CI/CD orchestrate pipeline workflows and promote code changes across environments in a controlled manner.

Container Registry

Container registries serve as centralized repositories for storing, managing, and distributing container images, facilitating efficient deployment and collaboration among development teams.

Virtual Network with Deployment in Private Subnet

Resources are deployed within a virtual network with a private subnet, enhancing security by isolating them from the public internet and other external networks.

Multi-Tenancy and Environment Management

Multi-Tenancy

In the platform architecture, there are two primary components:

the BDB control plane component and tenant-based components. The BDB control plane component serves as the central orchestrator, managing the overall platform infrastructure and resources.
On the other hand, the tenant-based components cater to the specific needs of individual tenants or customers. Each tenant operates within its own isolated environment, ensuring data privacy, security, and performance.

Logical Isolation

Tenants are logically isolated from each other, meaning that they share the same underlying infrastructure but operate as separate entities. This approach allows for efficient resource utilization while maintaining strict isolation between tenants. By leveraging Kubernetes namespaces, network policies, and RBAC, organizations can ensure that tenants' workloads remain segregated and secure.

Environment Management

The multi-tenancy model facilitates the creation of distinct development (Dev), staging (Stg), and production (Prod) environments within the same Kubernetes cluster. Each environment serves a specific purpose in the software development lifecycle, enabling teams to iterate, test, and deploy applications seamlessly. This consolidation of environments within a single cluster offers several advantages:

Centralized Management

Managing Dev, Stg, and Prod environments from a unified control plane simplifies operations and streamlines workflow processes. Teams can leverage consistent tooling, policies, and configurations across environments, enhancing collaboration and productivity.

Resource Efficiency

With a shared infrastructure model, teams can dynamically allocate resources based on workload demands across different environments. Kubernetes' elasticity and scalability features enable automatic scaling of resources to match application requirements, ensuring optimal performance and efficiency.

Simplified Maintenance

Maintenance tasks such as upgrades, patches, and monitoring can be performed centrally for all environments, reducing administrative overhead. Organizations can implement rolling updates and automated workflows to minimize downtime and ensure continuous availability of services.

Provisioning Infrastructure with CloudCLI, Web Console, and Terraform Scripts

CloudCLI

CloudCLI and web console interfaces provided by cloud service providers offer interactive and intuitive methods for provisioning and managing cloud resources. Through command-line interfaces (CLI) or web-based consoles, operators can execute commands or utilize graphical interfaces to create, configure, and monitor various services and components within the cloud environment. These tools provide real-time visibility and control over cloud resources, enabling efficient management and troubleshooting.

Web Console

CloudCLI and web console interfaces provided by cloud service providers offer interactive and intuitive methods for provisioning and managing cloud resources. Through command-line interfaces (CLI) or web-based consoles, operators can execute commands or utilize graphical interfaces to create, configure, and monitor various services and components within the cloud environment. These tools provide real-time visibility and control over cloud resources, enabling efficient management and troubleshooting.

Terraform Scripts

Terraform is an infrastructure-as-code (IaC) tool that enables declarative configuration and provisioning of cloud infrastructure. The platform utilizes Terraform scripts to define infrastructure resources, dependencies, and configurations in a human-readable format. By codifying infrastructure requirements, organizations can automate the deployment and management of resources across different cloud providers, ensuring consistency, repeatability, and scalability.

Multi-Cloud Support

The platform's infrastructure provisioning strategy includes Terraform scripts tailored for major cloud providers, such as AWS, Azure, Google Cloud Platform (GCP), and others. Each Terraform script is designed to provision resources specific to the target cloud environment, leveraging cloud provider APIs and services. This multi-cloud approach ensures portability and interoperability across different cloud platforms, enabling organizations to avoid vendor lock-in and leverage best-of-breed services from multiple providers.

BDB-Platform Installation steps

Client Tools Prerequisites

To start the platform deployment, below listed software need to be installed on local machine / Jump server.

Linux (If you are using windows, please use ubuntu WSL and install below software's)
Git - https://github.com/git-guides/install-git
Kubectl - https://kubernetes.io/docs/tasks/tools/
Flux - https://fluxcd.io/flux/installation/
Kubernetes cluster. Cluster should be ready and update your machine / jump server with kubeconfig. Kubectl command should point this cluster.
SOPS - https://fluxcd.io/flux/guides/mozilla-sops/

Platform installation

Once you have installed the Client Tools Prerequisites software on your local machine or jump server, you can start the deployment by following below steps.

1. Copy the FluxCD template from BDB git repository

$ git clone https://gitlab.bdb.ai/devops/bdb_clusters/bdb_release -b 8.6.0

2. Configure FluxCD with your git Repository

$ gitlab bootstrap command flux bootstrap gitlab --hostname=<gitlab.com> --token-auth --owner=<group/path> --repository=<project-name> --branch=main --path=/ --components-extra=image-reflector-controller,image-automation-controller

$ github bootstrap command flux bootstrap github --hostname=<github.com> --token-auth --owner=<group/path> --repository=<Project-name> --branch=<branch-name> --components-extra=image-reflector-controller,image-automation-controller -personal

3. Install the platform Prerequisites.

Memcached

ActiveMQ

mysql-audit (Optional)

Kafka

Metrics-Server

MongoDB

MySQL

Spark-Operator

mysql-repo

4. Install BDB-Platform components.

To install the BDB platform, you have to edit the /bdb-release/deployments/platform/release.yaml with proper details of your cluster.

Below information are required to fill this file:

URL of your image registry and login credentials if any.
Node Selector (optional) - Node label to schedule the pods to particular node pool.
Network file system (NFS/EFS) - Host and path.
Storage class if any specific
Volume Mount Type - persistentVolume by default
Namespaces(optional) - default values are configured
Single sign-on details (optional)
Domain name to access platform with SSL certificate(optional, but highly recommended).

5. Initialize platform and Configure the Admin setting

Once all the pods are up and running, you can create tenant(s) by enabling the tenant creation. This will run a job to create tenant and configure all the admin settings for the same.

Change the value of tenantConfig.enabled to true.
You need to increase the version number each time to run the job.
Separate namespace is required for each tenant. All the tenant based deployments service

Account and shared volume will be created from chart.

Like shared volume, configure the server and path for spark persistent volume. This is used for spark spill over during spark job execution.
Finally mount the tenant configuration file, which contains all the for the tenant and admin settings.

6. Enable the Monitoring system

For Enabling monitoring, Modify the Customization file in deployment/monitoring.

Metrics-Monitoring
Log-Monitoring

7. Basic sanity

You can enable the sanity check by changing the values to true. This will create a cronjob to on given schedule.

8. Content Migration

To migrate the sample content or promote the contents from DEV to STG or Prod, you can deploy the job to do the migration. Pre-requisites to this job are the container image which contains the contents to be migrated to this environment. You need to configure separate job for each tenant.

Check SPACEKEY and USERID are valid by login to BDB UI run development tools from the browser, click on networks tab and the response tab.

Connect with a BDB Expert

Connect Now