Setting up the Upload process on the DAM Portal

Overview

Importing assets in /portal is done through the following workflow:

1. Create a batch (job)

2. Upload items within this batch

   1. once uploaded, assets are analyzed (DAM_Utils)

   2. once analyzed, a workflow action is triggered by DAM_Utils on items to move them from the analyzing status to the next status. By default following status is one of:

      • rejected - for some reason, the item is rejected by the system.

      • duplicate - the asset is detected as a duplicate

      • index - the asset needs indexing

      • approved - the asset is technically valid for import.

3. User enrich metadata by selecting one or many items of the batch and filling in one or many properties.

4. Once user is satisfied with metadata enrichment of some items, he can validate them to start the final import to target collection.

Server configuration

Structures

Structures involved in mass-import process from the front are not meant to be accessed from the Back end for functional reasons. It is nevertheless quite helpful from a technical perspective. Therefore, structures involved in /portal mass-import process have been configured by default with a technical point of view. Standard configuration provided by the product NAR file should be sufficient in most cases. There should be no need in adding properties to those structures. If you think a property is missing, the product team will be pleased to analyze your requirement.

massimportjob

This structure holds data representing batches. Whereas extending batch with additional properties is possible, the portal application only offers the ability to configure specific medtadata at creation. If you need to be more specific in the way job metadata are managed, please submit a Jira ticket to product team.

Brief explanation on fields

• name: Not used by portal at this stage 11.28 will be used in future releases to manage assets orders.

• changes: Technical field. Do not change, do not edit, do not remove

• jobowner: Current user assigned to the batch. This value can be changed by transferring a job to another user

• expecteditems: Not used by portal at this stage 11.28 will be used in future releases to manage assets orders.

• defaultcollection: Default target collection for items.

• outstanding: Technical field. Do not change, do not edit, do not remove

• previousowner: Allows to know who transferred the job.

• activestep: Technical field. Do not change, do not edit, do not remove

• primaryjob: Technical field. Do not change, do not edit, do not remove

• jobprogress: Technical field. Do not change, do not edit, do not remove

massimportjob instances are often referred as jobs or batches.

massimportitem

This structure holds data representing a to be asset. This structure should contain all metadata that is computed by DAM_Utils plugin on an asset. Such metadata are usually not updatable by the user.

Brief explanation on fields

• binary: Hold the binary file of the to be asset

• job: Job to which belong the asset

• name: Name of the to be asset. This field should have the same configuration as the target field

• changes: Technical field. Do not change, do not edit, do not remove

• objectid: Technical field. Do not change, do not edit, do not remove (only used for assets mass update - out of scope of 11.28)

• objecttype: Technical field. Do not change, do not edit, do not remove (only used for assets mass update - out of scope of 11.28)

• changescache: Technical field. Do not change, do not edit, do not remove

• rejectreason: Fill in this field when you need to reject an item programatically to give feedback to users

• datachanges: (i18n) Technical field. Do not change, do not edit, do not remove. Make sure same i18n languages exist than all managed languages in your application

massimportitem instances are often referred as items.

Workflows

wkfmassimportjob

Workflow used by massimportjob. This workflow cannot be changed.

wkfmassimportitem

Workflow used by massimportitem. This workflow can be changed, but it must follow a few rules:

• it must include a step of id 2

• it must include a failed step.

• it must include an approved step (online, published)

• it must include a processing step

• it must include an imported step

• approved step must have an import action targeting processing step

• processing step must have a complete action targeting imported step

• processing step must have a fail action targeting failed step

• once the processing state is reached, the item is considered inaccessible → all business process should be running from id=2 included to processing step excluded.

Security

massimportjob

Standard security rules are applied:

• A user needs objectdata/insert right to create a job

• A user needs objectdata/update right to update a job (transferring a job for instance)

• A user needs objectdata/delete right to delete a job

• A user needs objectdata/changestatus rights to start a job (a job is started by invoking the start action)

massimportitem

Standard security rules are applied:

• A user needs objectdata/insert right to create an item

• A user needs objectdata/update right to update an item (transfer an item to another job)

• A user needs objectdata/delete right to delete an item

• A user needs objectdata/changestatus rights on all actions that are business oriented but also on technical actions that are not triggered by the user directly such as complete, fail…

Managing duplicates

Starting from 11.28, duplicates are only managed during the import phase. This change has been made in regards to following:

• In previous versions, when an asset was detected as a duplicate, if the user wanted to keep both, it was difficult to unmark an asset as a duplicate. Even more, false duplicates detected by the system would re-appear even if the user had marked them as non duplicates, if for some reason DAM_Utils rules were ran again

• We believe a user wants to be informed before adding an asset that it is a duplicate. If he decides to integrate the asset even though, he clearly is saying that whatever the system thinks, it is not a duplicate, or he wants to have a duplicate (and therefore doesn’t want to be bothered again about it)

In order to achieve this new behaviour, the duplicates field originally stored on each #damobject structure is now only available on massimportitem. Additionally, duplicates is a field holding a JSON representation of duplicates matching the massimportitem, but the information of an item being a duplicate is only presented to the user when the item is in the duplicate state.

Managing duplicates as a workflow step eases a lot process around duplicates management:

• As we are dealing with a workflow state, we can easily limit users that are allowed to approve an item detected as a duplicate

• It is possible to add business specific code to be triggered when an item reaches / leaves the duplicate workflow state.

PACKAGED_DAM_Utils configuration

Setting up dam import on portal requires you to review your DAM_Utils configuration. If you have not customized the plugin, the easiest way is to use the default configuration provided.

If you have changed the default configuration of the plugin, find below a change log of what has been done on the default configuration, and adapt your current DAM_Utils configuration

Before going further, if you are enabling massimport from a version prior to 11.28, make sure you have read Upgrading to 11.28

Additionally, it might be a good idea to make sure you are up to date with latest DAM_Utils changes Metadata and AI extraction with DAM_Utils as some concepts were added for massimport capabilities.

Parameters changes

dam_objects_selector

dam_objects_selector should now include massimportitem

dam_denormalization_config

aliases

Find below the standard aliases defined in PACKAGED_DAM_utils. This will help you update your actual configuration

presets

Specific presets have been defined:

preventExecutionFromMassImport This allows to skip DAM_Utils rules when an asset is created from a massimportitem. Please note that this preset requires the field tmpsource to exist on #damobject.

isDuplicate This allows to detect that an item is actually discovered as a duplicate. It is used as a guard in transformations running workflow actions

transformations

Make sure each root entry in the transformations array includes massimportitem. This will make DAM_Utils run transformations for massimportitem instances.

Additionally, add the preventGuard referenced by the preset preventExecutionFromMassImport is also included on each root entry. This will prevent DAM_Utils to run transformations again when a #damobject is created from a massimportitem

asynctransforms

As for transforms, make sure asyncTransformations acting on data are not re-run when a #damobjectis created from a massimportitem:

Add some additional asyncTransforms after the one(s) prevented by the preventExecutionFromMassImport guard:

A first one only running only on #damobject, to clear the tmpsource value received from the massimportitem. This will make sure tmpsource si cleared in the target collection after the massimportitem has been fully imported:

A second one running only on massimportitem, to force “massimport” as a value on tmpsource

A third one running only on massimportitem, to execute a workflow action that will make the item quit the draft (analyzing) status and reach a status valid according to your business rules:

A forth one running only on massimportitem to mark the asset as a duplicate.

Portal configuration

As of 11.28, configuration is scattered into two main domains. This will hopefully change in future releases.

Some entries were not cleaned from development phase. Please ignore files that are not listed

As a reminder, this doc refers to configuration keys in camelCase. Remember if you create files for keys to prefer the kebab-case as Uppercased filenames are not safe in a wedia application.

upload domain

this refers to everything within /upload config folder:

upload.defaultCollectionsOrder: string[] Allows to order the damCollections. This config is ignored if you choose to skip the default collection choose phase

upload.maxItemsPerJob: Number Allows to limit the number of items in a job. This is not a hard stop as a user can upload a zip containing a number of files that make him go over the limit. The limit is required because some filters during the indexing phase require in memory processes (cannot be performed through REST API calls). When reaching the index phase, portal needs to load all items contained in a batch. The higher you configure maxItemsPerJob, the longer the user waits before being able to work on the batch.

upload.skipChooseCollection: Boolean Allows to skip the choose default collection screen

upload.skipIndexing: Boolean If set to true, the massimport process will stop at upload phase. Once all items are uploaded and analyzed by DAM_Utils, the Validate this batch action will trigger the import process. This feature is likely to be improved in future releases.

upload.unzip.defaultValue: Boolean default checked state of the Automatically decompress zip files checkbox during upload

upload.unzip.exceptForEndsWith: string[] allows to ignore checkbox checked if the file name ends with one of given String in this array

upload.uploadJobDefaults: Object default properties values for creating jobs.

damImport domain

damImport.duplicates.authorizedResources: string[] List of resources that can be found as duplicates (list all asset types + massimportitem) If you don’t want to include massimportitem in it, you must change duplicates config in DAM_Utils. This configuration is likely to be removed in a future release

damImport.duplicates.display: Object Configures the way to display assets in the duplicates resolution modal.

damImport.duplicates.statusId: Number ID of the duplicate status

damImport.fieldsConfigOverload: Deprecated since 11.29 Object Each entry in the object is the name of a property for which you want to override default values. The value is an Object containing options. As of 11.28, the only available option is excelSearch String (only for childx field types) allows activate excel search on that field. You must give the name of the field that will be searched using excel.

damImport.fieldsDisplayPolicy: Deprecated since 11.29 “union“|”intersection”|”strictIntersection” see doc in _portal/dam-import/_.json

damImport.filters.defaultResourceConfig: Object Same as filters for standard search

damImport.hideSimulatedDataSelectionPrevNextButton:Deprecated since 11.29 Boolean see doc in _portal/dam-import/_.json

damImport.items.display.defaultConfig: Object Same as display configs for dam/explore

damImport.items.actions: String[] available actions for each item in the list

damImport.items.entityPreview.namedVariation: String name of the variation to use when hovering an asset in the small square layout

damImport.items.entityPreview.delayOpening: Number delay (in ms) before showing the hovering preview

damImport.items.rootNatures: Icons to display based on root assetnature

damImport.layouts: Object same as layouts in dam/explore

damImport.namingConventions: Object | null allows to confure a bloc to display a warning below upload zone to inform users about naming conventions.

The Object must have the following structure:

damImport.pollingInterval: Number Interval for checking for running commands

damImport.simulatedDataSelectionAutoChangePictureInterval: Deprecated since 11.29Numbersee doc in _portal/dam-import/_.json

damImport.upload: From

damImport.uploadPollingSettings.baseTime: Number Used to define a polling interval for checking if uploads and analyzing processes are completed

damImport.uploadPollingSettings.itemLoadingAverage: Number used to adapt polling time between 2 calls.

damImport.workflow.hiddenActions: String[] actions that should be excluded from workflow actions menus (reference technical actions that should not be used by users)

damImport.workflow.statusesConfig: ObjectAllows to override default server-side configured statuses (colors, labels) and to add icons

Handling deprecated configs when migrating from 11.28 to 11.29

damImport.fieldsConfigOverload → Configuration of fields is common to the unitary edit config.

damImport.fieldsDisplayPolicy → no fallback, Product displays intersection of fields, with no choice offered anymore

damImport.hideSimulatedDataSelectionPrevNextButton → no fallback, the targeted component for this config is not available in 11.29 and will be used for different purpose in further versions

damImport.simulatedDataSelectionAutoChangePictureInterval → no fallback, the targeted component for this config is not available in 11.29 and will be used for different purpose in further versions

REST API Configuration

In order for portal to be able to retrieve what fields are editable on target structures, all #damobject structures MUST have their update service enabled.

Mass imports/uploads