Skip to main content

Import products to Commerce Layer

Context​

We need to import web channel products into Commerce Layer, required product attributes are:

  • SKU
  • Name (Must be unique in Commerce Layer)
  • Shipping Category

It is desirable to reuse and take advantage of as much of the current infrastructure as possible.

Commerce Layer applies rate limits on their API, so we need to do as less request as possible in a short period of time. This can require to group product creation messages. Commerce Layer core API rate limits

Import endpoints on Commerce Layer core API also have the following limits: Import limits

Decision​

We will create more or less similar integration as the one done for Algolia project, reusing current createproduct event type from deduplicationqueue source in Product Eventbridge bus.

Design for the process to import products into Commerce Layer

  1. New event rule needs to be created to filter that events and process only new web channel products.
    • Nav to add new AvailableForChannel field on product xml to identify product channel.
  2. Modify Nav Product lambda and Platform Product lambda to handle AvailableForChannel attribute.
  3. Subscribe to the new rule a new FIFO SQS to process messages in order.
  4. Create new product processing lambda to read the queue, transform incoming Nav data into the format needed for Commerce Layer and write in S3 a json file per each product.
    • Product name needs to be transformed to ensure it will be unique in Commerce Layer.
    • Shipping Category doesn't come from Nav data, lambda needs to setup a default value.
    • If a file already exists, the lambda should overwrite it with the new data (To ensure that in the S3 is the last product modification).
  5. Create the required S3 bucket to store product creation files.
  6. Create a new lambda triggered by a cron expression to process S3 bucket files. It needs to read all the current existing files in the S3 and send a single import request to Commerce Layer, then delete the processed files from the S3

Consequences and limitations​

  • S3 bucket can contain lots of product files, not sure how it can impact the performance or pricing.
  • If there is an error on the import process, as it is asyncronous, and the files in the S3 are deleted, it makes dificult to perform automatic retries.

Resources​