> For the complete documentation index, see [llms.txt](https://host2host.onibonje.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://host2host.onibonje.com/docs/07-multi-country-deployment.md).

# Multi-Country Deployment

## 1. Overview

The Bank operates across **20+ African countries** with a standardized Finacle core banking environment. The H2H platform must deploy consistently across all markets while accommodating country-specific regulatory, operational, and calendaring requirements — without forking the codebase per country.

## 2. Deployment Principles

| Principle              | Implementation                                    |
| ---------------------- | ------------------------------------------------- |
| Standardized platform  | Same JAR modules and runtime across all countries |
| Localized behavior     | Country differences via database configuration    |
| Regional resilience    | Active-active deployment in multiple regions      |
| Centralized governance | Global admin portal with country-scoped RBAC      |
| Distributed execution  | Transaction processing runs in regional clusters  |
| Data residency         | Transaction data stays in-region where required   |

## 3. Regional Topology

```mermaid
flowchart TB
  subgraph global [Global Control Plane]
    Admin[Admin Portal]
    ConfigDB[(Global Config DB)]
    Artifactory[JAR Repository]
    Vault[Vault Cluster]
    Keycloak[Keycloak]
  end

  subgraph regionA [Region A - West Africa]
    GW_A[H2H Gateway]
    RT_A[Runtime Pods]
    Kafka_A[Kafka Cluster]
    DB_A[(Regional TX DB)]
    Finacle_A[Finacle Instance]
  end

  subgraph regionB [Region B - East Africa]
    GW_B[H2H Gateway]
    RT_B[Runtime Pods]
    Kafka_B[Kafka Cluster]
    DB_B[(Regional TX DB)]
    Finacle_B[Finacle Instance]
  end

  Admin --> ConfigDB
  global --> regionA
  global --> regionB
  GW_A --> RT_A --> Finacle_A
  GW_B --> RT_B --> Finacle_B
  ConfigDB -.->|config sync| RT_A
  ConfigDB -.->|config sync| RT_B
```

### 3.1 Region Assignment

Countries are assigned to regions based on latency, data residency, and Finacle deployment:

| Region          | Example Countries | Finacle Instance   |
| --------------- | ----------------- | ------------------ |
| West Africa     | NG, GH, SL, LR    | Finacle WA cluster |
| East Africa     | KE, TZ, UG, RW    | Finacle EA cluster |
| Southern Africa | ZA, ZM, MW        | Finacle SA cluster |
| Central Africa  | CD, CM            | Finacle CA cluster |

Exact mapping is defined in deployment configuration, not in code.

## 4. What Is Shared vs Localized

### 4.1 Shared Across All Countries

| Asset                                   | Storage           |
| --------------------------------------- | ----------------- |
| JAR modules (runtime capabilities)      | Artifactory       |
| Route template library (skeleton flows) | Global config DB  |
| Step type registry                      | JAR code          |
| Admin portal application                | Global deployment |
| Canonical message models                | `h2h-common` JAR  |
| Monitoring dashboards (templates)       | Grafana           |
| ISO 20022 / MT940 parsers               | Transform JARs    |

### 4.2 Localized Per Country

| Asset                                 | Storage                       |
| ------------------------------------- | ----------------------------- |
| Partner profiles                      | Config DB (country-scoped)    |
| Channel configurations                | Config DB                     |
| Transform specs (country CSV formats) | Config DB                     |
| Validation and routing rules          | Config DB                     |
| Holiday calendars                     | Config DB                     |
| Cut-off schedules                     | Config DB                     |
| Finacle service endpoints             | Environment config per region |
| Regulatory field requirements         | Config DB (validation rules)  |

### 4.3 Localized Per Region (Infrastructure)

| Asset                   | Configuration                   |
| ----------------------- | ------------------------------- |
| Kubernetes cluster      | Per region                      |
| Kafka cluster           | Per region                      |
| SFTP gateway endpoints  | Per region                      |
| Transaction metadata DB | Per region                      |
| Finacle connection      | Per region                      |
| Vault namespace         | Per region (with global policy) |

## 5. Country Pack Deployment

### 5.1 Preferred: Configuration Pack

A **country pack** is a bundle of published database configuration — not a code deployment:

```
country-pack-NG/
├── partners/              # seed partner configs (if migrating)
├── transforms/            # NG-specific CSV mappings
├── rules/                 # NG regulatory validation rules
├── calendars/             # Nigeria public holidays
├── schedules/             # WAT timezone cut-offs
└── manifest.json          # pack metadata, version, dependencies
```

Deployed via admin API or migration script. No JAR changes.

### 5.2 Exception: Country Route JAR

Create `h2h-routes-{cc}` JAR only when a country requires a **new step type** that cannot be expressed as configuration:

```yaml
# Enable in regional runtime
h2h:
  modules:
    routes-ng:
      enabled: true
```

This should be the exception, not the default.

## 6. Configuration Synchronization

```mermaid
sequenceDiagram
  participant Admin as Admin Portal
  participant GlobalDB as Global Config DB
  participant Kafka as Kafka (global)
  participant RegA as Region A Runtime
  participant RegB as Region B Runtime

  Admin->>GlobalDB: Publish NG partner config
  GlobalDB->>Kafka: CONFIG_PUBLISHED (DB-resolved channel)
  Kafka->>RegA: Invalidate cache (NG scope)
  Note over RegB: No action (not in scope)
  RegA->>GlobalDB: Reload NG config
```

* **Global config DB** is the source of truth for all configuration
* Regional runtimes cache only config relevant to their assigned countries
* Cache invalidation events are scoped by country code
* Regional runtimes filter: `WHERE country_code IN (:assignedCountries)`

## 7. Kubernetes Deployment

### 7.1 Helm Chart Structure

```
helm/h2h-platform/
├── Chart.yaml
├── values.yaml                    # defaults
├── values-west-africa.yaml        # region overrides
├── values-east-africa.yaml
├── templates/
│   ├── runtime-deployment.yaml
│   ├── admin-deployment.yaml
│   ├── gateway-deployment.yaml
│   ├── kafka.yaml
│   ├── configmap.yaml
│   └── secrets.yaml
└── country-packs/
    ├── ng-values.yaml             # NG-specific config flags
    └── gh-values.yaml
```

### 7.2 Runtime Deployment per Region

```yaml
# values-west-africa.yaml
region: west-africa
assignedCountries: NG,GH,SL,LR
finacle:
  endpoint: https://finacle-wa.internal.bank.com
kafka:
  bootstrapServers: kafka-wa.bank.com:9092
replicaCount: 3
autoscaling:
  minReplicas: 3
  maxReplicas: 10
  targetCPUUtilization: 70
```

### 7.3 Active-Active Requirements

| Requirement                | Implementation                                   |
| -------------------------- | ------------------------------------------------ |
| No single point of failure | Minimum 3 replicas per service                   |
| Cross-AZ deployment        | Pod anti-affinity across availability zones      |
| Kafka replication factor   | 3 minimum                                        |
| Database                   | PostgreSQL with streaming replication            |
| Health checks              | Liveness and readiness probes on all pods        |
| Graceful shutdown          | Camel context shutdown with in-flight completion |

## 8. Environment Strategy

| Environment | Purpose                    | Finacle Target     |
| ----------- | -------------------------- | ------------------ |
| Development | Developer testing          | Finacle dev/mock   |
| Sandbox     | Partner onboarding testing | Finacle sandbox    |
| UAT         | User acceptance testing    | Finacle UAT        |
| Production  | Live processing            | Finacle production |

Each environment has:

* Separate Kubernetes namespace
* Separate config DB schema (or environment column filter)
* Separate Vault namespace
* Separate Kafka cluster — or regional `event_channel_def` rows with `scope_type: REGION` and distinct `destination_name` (see [46 Database-Driven Events](/docs/46-database-driven-events.md))

**Sandbox configs** (`environment = sandbox`) are never loaded by production runtime.

## 9. Rollout Strategy for New Countries

```mermaid
flowchart LR
  A[Deploy regional infra] --> B[Import country pack]
  B --> C[Configure Finacle endpoints]
  C --> D[Onboard pilot partner]
  D --> E[Sandbox testing]
  E --> F[UAT sign-off]
  F --> G[Production go-live]
  G --> H[Monitor and stabilize]
```

| Phase                  | Activities                                       | Duration Estimate |
| ---------------------- | ------------------------------------------------ | ----------------- |
| 1. Infrastructure      | Deploy K8s, Kafka, DB in region (if new region)  | 2-4 weeks         |
| 2. Country pack        | Import transforms, rules, calendars              | 1 week            |
| 3. Finacle integration | Configure FCJ/FCUBS endpoints, test connectivity | 1-2 weeks         |
| 4. Pilot partner       | Onboard one corporate client in sandbox          | 2 weeks           |
| 5. UAT                 | End-to-end testing with pilot partner            | 2 weeks           |
| 6. Go-live             | Publish production config, monitor               | 1 week            |

**Target:** New country onboarding in **4-6 weeks** after infrastructure is in place (vs months with bespoke integrations).

## 10. Disaster Recovery

| Scenario           | Recovery                                                     |
| ------------------ | ------------------------------------------------------------ |
| Single pod failure | K8s auto-restart, no data loss                               |
| AZ failure         | Remaining AZ replicas continue; Kafka ISR                    |
| Region failure     | Failover to secondary region (if cross-region DR configured) |
| Config DB failure  | Restore from backup; runtimes use cached config              |
| Kafka failure      | Messages buffered at gateway; replay on recovery             |

**RPO:** < 1 minute (Kafka replication) **RTO:** < 15 minutes (K8s auto-healing)

## 11. Related Documents

* [Architecture Overview](/docs/01-architecture-overview.md)
* [Database-Driven Configuration](/docs/04-database-driven-configuration.md)
* [Personas and RBAC](/docs/06-personas-and-rbac.md)
* [Implementation Roadmap](/docs/10-implementation-roadmap.md)


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://host2host.onibonje.com/docs/07-multi-country-deployment.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.