How to Transfer Data

Introduction

Exported data is placed in a dedicated Google Cloud Storage (GCS) bucket. This document explains the available methods for securely accessing and transferring your MCM data export to your own cloud environment.

MCM supports several access methods depending on your cloud provider and security requirements. Choose the method that best fits your organization's infrastructure and compliance needs.

Access Methods Overview

MethodCloud ProviderCredential TypeKey ManagementBest For
Dynamic Key Rotation (API)AWS, Azure, or other non-GCPShort-lived (Max 48 hours) HMACAutomated via APICustomers using S3-compatible libraries who want automated credential rotation
Workload Identity FederationAWS, Azure, GCPKeyless (federated identity)No keys requiredEnterprise customers prioritizing security and compliance
GCP Storage Transfer ServiceGCPGoogle-managed service agentManaged by GoogleCustomers already on GCP
📘

NOTE

While GCP supports Workload Identity Federation, GCP Storage Transfer Service is the recommended method.

🚧

IMPORTANT!

To enhance data security and compliance, MCM is transitioning customers using non-GCP cloud providers to one of the two new access methods:

1. Workload Identity Federation OR 2. Dynamic Key Rotation

These methods replace the legacy approach of manually sharing long-lived credentials and provide significantly improved security through automated rotation or keyless authentication.

If you are currently using a static HMAC key or service account key, please contact your MOLOCO representative to begin migrating to one of the new methods. Data export access using the legacy manual method will no longer be supported after October 1, 2026.

1. Workload Identity Federation (Recommended)

Workload Identity Federation (WIF) is the most secure access method available. It enables keyless authentication, allowing your AWS or Azure workloads to access the GCS export bucket directly without ever handling a secret key. This is achieved by establishing a trust relationship between your cloud identity provider and Google Cloud.

Prerequisites

  • An AWS IAM Role (or Azure equivalent) dedicated to accessing the MCM export bucket.
  • The ability to configure your IAM Role's Trust Relationship.
  • The Google Cloud SDK (google-cloud-storage v1.27.0 or later for Python).

How it works?

  1. MOLOCO configures a Workload Identity Pool in GCP and adds a provider linked to your cloud identity (e.g., AWS account).
  2. MOLOCO provides you with a Credential Configuration file (gcp-creds.json) that contains the metadata needed to bridge your cloud environment with GCP.
  3. Your application uses the Google Cloud SDK, which automatically performs the identity token exchange in the background — no HMAC keys or service account JSON files are involved.

What benefits do you get?

BenefitDescription
Zero secretsNo keys are generated, stored, or rotated. Eliminates the risk of credential leakage entirely.
Industry standardFollows the OpenID Connect (OIDC) federation model, a widely adopted standard for cross-cloud authentication.
Reduced operational overheadNo credential rotation or manual key management required.

Setup Steps

STEP 1: Identify your Workload Identity

Create (or identify an existing) IAM Role in your cloud environment that will be used to read from the export bucket. For example, in AWS, this could be a role assumed by Lambda, ECS, EC2, or another compute service.

STEP 2: Provide Your Identifier to Moloco

Share the Full Role ARN (for AWS) with your Moloco representative.

arn:aws:iam::123456789012:role/my-data-export-role

For Azure, provide the equivalent identity information as requested by your Moloco representative.

STEP 3: Prepare Your IAM Role (AWS Example)

Ensure your role's Trust Relationship policy allows the service that will run your data pipeline to assume the role. For example, for an AWS Lambda function:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "sts:AssumeRole",
            "Principal": {
                "Service": "lambda.amazonaws.com"
            }
        }
    ]
}

STEP 4: Integrate the Credential Configuration

Moloco will provide a Credential Configuration file (e.g., gcp-creds.json). Place this file in your application's runtime environment and set the following environment variable:

export GOOGLE_APPLICATION_CREDENTIALS="/path/to/gcp-creds.json"
📘

NOTE

Use the official Google Cloud SDK (not a generic S3 library) for Workload Identity Federation. The SDK handles the identity token exchange automatically.

STEP 5: Access the Bucket

Using the Google Cloud Storage client library, your application can access the export bucket natively. The SDK automatically reads the credential configuration and performs the cross-cloud token exchange in the background.

Python Example (e.g., AWS Lambda):

import os
from google.cloud import storage

def lambda_handler(event, context):
    os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = "gcp-creds.json"

    try:
        client = storage.Client()
        bucket_name = "mcm-export-your-platform-name"

        bucket = client.get_bucket(bucket_name)
        blobs = list(client.list_blobs(bucket_name, max_results=5))

        print(f"Successfully connected to bucket: {bucket_name}")
        print(f"First {len(blobs)} objects:")
        for blob in blobs:
            print(f"  - {blob.name}")

        return {
            "statusCode": 200,
            "body": f"Connected! Found {len(blobs)} objects."
        }

    except Exception as e:
        print(f"ERROR: {str(e)}")
        return {"statusCode": 500, "body": str(e)}
📘

NOTE

MOLOCO provides the GCP-side configuration (Workload Identity Pool, provider, and service account impersonation). You are responsible for managing your own cloud environment, including the IAM Role and its permissions.

2. Dynamic Key Rotation (API)

Dynamic Key Rotation provides automated, short-lived HMAC credentials through the MCM Management API. Your application calls the RotateDataExportKey API endpoint to obtain fresh credentials before each data access session. Each call invalidates all previously issued keys and returns a new HMAC key pair.

Prerequisites

  • Access to the MCM Management API. You must have a valid API key with ADMIN permissions for your platform. If you have not yet onboarded to the Management API, please contact your MOLOCO representative.

How it works?

  1. Your application calls the RotateDataExportKey endpoint on the MCM Management API.
  2. The API invalidates all previously issued HMAC keys for your platform's service account.
  3. A new HMAC key pair (key_id and secret_key) is returned.
  4. Your application uses the new credentials to access the GCS export bucket via any S3-compatible library or tool.

What benefits do you get?

BenefitDescription
DataEnhanced securityCredentials are effectively short-lived, minimizing the exposure window.
Automated rotationNo manual credential management or handoff required.
S3 compatibilityWorks with existing S3-compatible tooling and libraries; only the credential retrieval step is added.
📘

IMPORTANT NOTES

  1. You must call RotateDataExportKey to refresh credentials before every access to the export bucket.
  2. Each API call must have at least a 1-minute interval from the previous call.
  3. Newly issued credentials may take up to 60 seconds to propagate and become active. Plan for this delay in your integration.
  4. The secret_key is returned only once at creation time and cannot be retrieved again. Ensure your application stores it securely for the duration of the data transfer session
  5. Each call deletes all previously issued keys. Only the most recently issued key pair is valid at any time.

API Reference

Endpoint:

POST /rmp/mgmt/v1/platforms/{platform_id}/data-export/key

Headers:

HeaderValue
x-api-keyYour MCM Management API key
Content-Typeapplication/json

Response:

FieldTypeDescription
key_idstringThe unique identifier for the newly created HMAC key.
secret_keystringThe raw HMAC secret key. Only returned once upon creation.
📘

NOTE

For full API details, see the MCM Management API Reference.

Integration Example (Python)

The following example demonstrates how to retrieve fresh credentials and use them to access your export bucket.

STEP 1: Retrieve new HMAC credentials

import requests

PLATFORM_ID = "YOUR_PLATFORM_ID"
API_KEY = "YOUR_API_KEY"
BASE_URL = "https://YOUR_PLATFORM-mgmt.rmp-api.moloco.com"

url = f"{BASE_URL}/rmp/mgmt/v1/platforms/{PLATFORM_ID}/data-export/key"

headers = {
    "x-api-key": API_KEY,
    "Content-Type": "application/json"
}

try:
    response = requests.post(url, headers=headers)
    response.raise_for_status()

    data = response.json()
    key_id = data.get("key_id")
    secret_key = data.get("secret_key")

    print(f"Key ID: {key_id}")
    print("Secret Key: (stored securely)")

except requests.exceptions.HTTPError as err:
    print(f"HTTP Error: {err}")
except Exception as e:
    print(f"Unexpected error: {e}")

STEP 2: Access the GCS bucket with the retrieved credentials

Use the key_id as the Access Key and secret_key as the Secret Key with any S3-compatible library or tool. Since HMAC keys are compatible with the S3 API, you only need to change the endpoint from AWS S3 to GCS (storage.googleapis.com).

For example, using AWS DataSync or S3-compatible SDKs, configure:

3. GCP Storage Transfer Service

Customers using GCP can transfer data natively using Google Cloud Storage Transfer Service. This method uses a Google-managed service agent — no HMAC keys or service account files are needed.

Prerequisites

  • Provide your Storage Transfer Service agent email to your MOLOCO representative so that read access can be granted to your export bucket. The service agent email follows this format:
project-PROJECT_NUMBER@storage-transfer-service.iam.gserviceaccount.com
  • A destination GCS bucket in your project.

HMAC Key — Static (Legacy)

🚧

Legacy Method

Moloco no longer offers these legacy methods. Existing platforms which use this legacy method migrate to Dynamic Key Rotation or Workload Identity Federation . Please contact your MOLOCO representative to discuss migration options.

Choosing the Right Method

Use the following decision guide to select the best access method for your organization:

  • Already on GCP? → Use GCP Storage Transfer Service. No credentials to manage.
  • On AWS/Azure and prioritize security? → Use Workload Identity Federation.
  • Zero secrets, industry-standard federated identity. On AWS/Azure and prefer S3-compatible tooling?→ Use Dynamic Key Rotation. Minimal integration change with automated credential rotation.
  • Currently using static HMAC keys? → Contact your MOLOCO representative to begin migrating to one of the new methods above.

Next steps

  • For questions about data export access methods or migration, please contact your MOLOCO representative or see the MCM FAQ.