Write and Validate Custom Scripts

Write custom scripts for batch data transforms in a standard Python environment by using your preferred IDE and validate them locally against a sandbox.

Edition Table
Available in: Developer, Enterprise, Performance, and Unlimited Editions. See Data 360 edition availability.
Permission Sets Needed
To write custom scripts and validate them locally against a sandbox:Permission set:
  • Data Cloud Architect

  1. Set up your Data 360 environment.

    1. Verify that you have access to a Data 360 sandbox with appropriate permission sets. If you don’t have a sandbox, create one. See Create a Data 360 Sandbox.

    2. Enable Code Extension in Data Cloud Setup using the Feature Manager. See Enable Data 360 Features in Feature Manager.

    3. Verify that you ingested data into your sandbox to create the DLOs or DMOs that your script references.

  2. Set up Salesforce CLI for code extension and create a script package. For more information, see Set Up Salesforce CLI for Code Extension.

  • Work exclusively with either DLOs or DMOs in a single script.
  • Add any additional Python dependencies to requirements.txt.
  • Use only pip-installable dependencies. Libraries that require OS-level installations or system configurations don’t work in Data 360.
  • Use clear function names and comments, and handle errors gracefully.
  • Test your logic frequently in your selected environment.
  • Use logging for debugging and monitoring.

Use exact API names for Data 360 object fields. Field names follow the FieldName__c pattern.

Use one of these write modes when writing to DLOs or DMOs in your script.

Write ModeDescriptionExample
APPENDAdds new records to the target object without affecting existing data.client.write_to_dlo("Target_DLO__dll", df, write_mode=WriteMode.APPEND)
OVERWRITEReplaces all existing data in the target object with new data.client.write_to_dlo("Target_DLO__dll", df, write_mode=WriteMode.OVERWRITE)
MERGEUpdates existing records and adds new records based on key fields.client.write_to_dlo("Target_DLO__dll", df, write_mode=WriteMode.MERGE)

  1. Go to your code extension package folder.

  2. Create and activate a Python virtual environment in the package folder.

    macOS and Linux

    Windows (PowerShell)

    Windows (Command Prompt)

  3. Add required dependencies to requirements.txt.

  4. Install the dependencies.

    macOS and Linux

    Windows

To add new dependencies during development, always add them to requirements.txt and then install by using this command. This process ensures that all dependencies are included in your deployment package.

  1. Write your script in payload/entrypoint.py to reference the ingested data.

  2. Test your script.

    • --entrypoint <path_to_entrypoint> (required): Path to your script entry point (typically ./payload/entrypoint.py in the default package layout).
    • --target-org <org_alias> (required): The alias against which you’re testing your script.

    Example:

  3. Optionally, control the number of rows returned during local testing.

    By default, Data 360 queries return 1,000 rows when you test locally. To validate against a larger dataset, raise the row limit. To iterate faster on a smaller sample, lower it. Set the row limit in entrypoint.py before your script reads any data.

    1. Import config in your entrypoint.py.

    2. Set the row limit to the number of rows you need. For example, to return 5,000 rows:

      To remove the limit entirely and fetch all records, set the value to None:

    If you package and upload your entrypoint with this setting, the package includes the row limit, which then applies when run in Data 360. Remove or reset the row limit before packaging for deployment.

  4. Update the Data 360 configuration file after changing your script.

    The sf data-code-extension script scan command doesn’t work as intended for DMO-to-DMO transforms. You must manually craft the config.json file. See Configure config.json for DMO-to-DMO Transforms.

Deploy a Custom Script to Data 360 Sandbox by Using UI.