Skip to main content

Spike - Review Cloudinary asset migration from Amplience

Overview​

Results from a spike looking into migrating assets from Amplience to Cloudinary. The aim of the spike is to review different migration plannings and come up with possible solutions so we can move forward. How to migrate from Amplience's flat structure to a foldered structure with Cloudinary?

Migration Planning​

There are several migration options available:

  • Migration all existing content in one phase: This requires uploading all our media files. This option fits one of the following use cases:

    • All of our images and videos are actively used.
    • We intend to shut down our existing media storage.

    Implement this migration by creating a script that runs on our media assets and for each file it calls the upload API call.

      cloudinary.v2.uploader.upload("image_path", {public_id: "own_public_id"},
      function(error, result) { console.log(error,result) });

  • Lazy migration: Uploading a media file only when it is requested by our website or app user for the first time. This option is effective when we have a long tail of media assets, not all of them are active, and we are not sure which ones are still in use.

    Using the Cloudinary management console or the API, we can define a mapping between a folder name in our Cloudinary media library and a base remote URL, which links to our images online folder.

      cloudinary.v2.api.create_upload_mapping('cloudinary_folder_name',
      { template: "images_available_at" },
      function(error, result) { console.log(error,result) });

  • Hybrid approach: Run a script to upload the “hot” group of our most commonly used media assets and use the “lazy migration” option to upload the rest. This option works best when we have a defined subset of our media assets that drives most of our traffic.

  • Fetch assets: Fetch media assets from remote locations and store them for a predefined period. Use this option when our images and videos originate from various online sources and they are used for a short timespan, as in news items.

  • Uploading large files: If we are uploading files larger than 100MB, there is an option to do a chunked upload:

      cloudinary.v2.uploader.upload_large("my_large_image.tiff",
          { resource_type: "image", chunk_size: 6000000 },
          function(error, result) { console.log(error,result) });

Which file to upload​

Cloudinary is able to manipulate images and videos on-the-fly upon request or upon upload, so we only need to upload the highest resolution of one image or video. There is no need to upload large/medium/small variants.

Setting an Upload Policy Using an Upload Preset​

A convenient way to create a centralized upload policy is defining an upload preset. This enables us to define the transformations we would like to do once, then use the preset name to activate it upon upload. We can define several upload presets and use them according to different policies you have.

When we define an upload preset, we can set a transformation that will change the original file and then only the transformed file will be stored. This option is called an incoming transformation. We can also define transformations that will be created as derived files, which will be stored in addition to the original file. This process is called an eager transformation. Using Cloudinary, we can transform the images and video on-the-fly, therefore these options are required for cases where we would like to process the transformation immediately upon upload.

As an example, the following code creates an upload preset that adds the tag remote. The unsigned parameter determines if the preset can be used for unsigned uploads, which can be done from the client side without having the API secret. The allowed_formats parameter defines the file formats allowed to be used with this preset.

cloudinary.v2.api.create_upload_preset({ name: "my_preset",
    unsigned: true, tags: "remote", allowed_formats: "jpg,png" },
    function(error, result) { console.log(error,result) });

The following code uploads an image using this upload preset:

cloudinary.v2.uploader.upload("file_name.jpg",
    { public_id: “own_public_id”, upload_preset: "my_preset" },
    function(error, result) { console.log(error,result) });

Delivery Profile and Mapping function​

A delivery profile identifies a set of media assets by their delivery URL, and defines how they are delivered based on the source of the original media and applied transformations. Delivery profiles ultimately bring together all the different configuration components.

We will need to create a mapping function in order to translate Amplience coupled URLs to Cloudinary. Mapping functions enable Media Optimizer to apply the same transformations when delivering the media from its original source. Mapping functions can then be selected when creating a new delivery profile in the Media Optimazer sub-account in Cloudinary. Other static transformation to the URLs can also be added such as f_auto,q_auto, etc...

Taking the below code as an example:

https://rapha-res.cloudinary.com/image/fetch/q_auto,f_auto/https://i1.adis.ws/i/rapha/image_name?w=600,h=600

A mapping function will need to be created in order to translate ?w=600,h=600 into Cloudinary syntax.

Resources​