Data sources and access

Choose the data source and authentication method that best suits your business needs. Contact your Moloco representative for more information regarding catalog feed integration.

📘

Note

If you want to use an IP allowlist for data security purposes, contact your Moloco representative for the required IP addresses. We support HTTP Authentication and IP Allow List, but not both at the same time.

Amazon S3

If your team uses AWS, you can set up a shared S3 bucket by following this procedure. By creating a new empty S3 Bucket, in addition, you gain comprehensive control of:

• Determining which files are intended for sharing with Moloco
• Specifying the individuals and machines authorized to read from or write to the S3 Bucket

Step 1. Create a new Amazon S3 Bucket

If your platform is MyCommerce and you wish to share the production dataset, create an S3 Bucket named similar to my-commerce-prod-to-moloco-shared-bucket in your preferred AWS Region.

Step 2. Configure the S3 Bucket Policy

Set up the S3 Bucket Policy to provide read access for Moloco's AWS Account to your S3 Bucket. The Bucket Policy is in JSON below. Please contact us for Moloco's AWS Account ID information.

{
	"Version": "2012-10-17",
	"Statement": [
		{
			"Sid": "Read permissions for Moloco's ETL processor",
			"Effect": "Allow",
			"Principal": {
				"AWS": "arn:aws:iam::MOLOCO_AWS_ACCOUNT_ID:user/Moloco-MCM-Reader"
			},
			"Action": [
				"s3:List*",
				"s3:Get*"
			],
			"Resource": [
				"arn:aws:s3:::my-commerce-prod-to-moloco-shared-bucket",
				"arn:aws:s3:::my-commerce-prod-to-moloco-shared-bucket/*"
			]
		}
	]
}

📘

Note

We recommend to configure the S3 Bucket to use server-side encryption with Amazon S3 managed keys (SSE-S3), as it's easier to configure and test. But if your organization requires to use server-side encryption with AWS KMS keys (SSE-KMS), please use the below bucket policy instead of the above.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowListActionsOnS3Bucket",
            "Effect": "Allow",
            "Action": [
                "s3:List*",
                "s3:Get*",
            ],
            "Resource": [
                "arn:aws:s3:::arn:aws:s3:::my-commerce-prod-to-moloco-shared-bucket",
                "arn:aws:s3:::arn:aws:s3:::my-commerce-prod-to-moloco-shared-bucket/*",
            ],
            "Principal": {
                "AWS": "arn:aws:iam::MOLOCO_AWS_ACCOUNT_ID:user/Moloco-MCM-Reader"
            }
        },
        {
            "Sid": "AllowDecryptOnKMSKey",
            "Effect": "Allow",
            "Action": [
                "kms:Decrypt",
                "kms:GenerateDataKey"
            ],
            "Resource": "arn:aws:kms:YOUR_AWS_REGION:YOUR_AWS_ACCOUNT_ID:key/your_kms_key_id",
            "Principal": {
                "AWS": "arn:aws:iam::MOLOCO_AWS_ACCOUND_ID:user/Moloco-MCM-Reader"
            }
        }
    ]
}

The AWS KMS Key Policy Overview provides additional information on setting up and managing AWS Key Management Service (KMS) key policies.

Look at this AWS Knowledge Center article to learn more about configuring S3 bucket access and default encryption. These resources provide step-by-step guides and best practices for handling access permissions and encryption in AWS.

Step 3. Upload catalog file to the S3

Upload the Refresh Feed or Update Feed files to S3. For example, place the full update feed in the following S3 path:

s3://my-commerce-prod-to-moloco-shared-bucket/catalog-item/full-update/latest.csv.gz

Step 4. Provide Moloco with the S3 file location

After uploading the file and configuring the daily data export pipeline, please inform your Moloco representative of the S3 file location and the export schedule. Moloc will then establish the daily data import pipeline.