Data Loader – FTP Sites

Owned by Bettina Lippisch, created
Last updated: May 30, 2024
6 min read

Knowledge Base Home

Search the Omeda Knowledge Base

Data Loader allows users to load contact data into their Omeda database.  The loaded data then becomes a selectable segment in Audience Builder.

The FTP Sites section of Data Loader automates the file ingestion process and will look for files meeting specified criteria to pull in from the Omeda FTP site and process nightly.

Note: This feature requires additional permissions to be turned on. If you do not yet have access to this feature, please submit a support ticket.

FTP Site Setup – Default

Clicking on ‘FTP Sites’ in the Data Loader menu will bring you to the list of FTP sites connected to your account. The Omeda SFTP site has already been loaded into your account and you will just need to set the status to ‘Active’ (from the Actions drop-down) in order for the tool to being running.

image-20240520-165013.png

Third-Party FTP Site Setup

To add additional FTP sites to Data Loader, click ‘Create a Site’ at the top right side of the site list.

image-20240520-165020.png

This will open a popup that prompts you to name the new site:

Once you’ve named your new site and clicked ‘Create FTP Site’ you will be taken to the setup page:

For a successful setup, you will need the following things:

  • Host (domain of the server without protocol or port – e.g. sftp.omeda.com)

  • Username

  • Password

  • Protocol (defaults to ‘SFTP‘)

  • Port

FTP File Setup and Navigation

To manage email lists, setup Data Loader tracking jobs, or view the general site details, select “Edit Site Settings’ from the Actions dropdown.

To see a list of the current files available on the directory, choose “View Site’s Directory”.

FTP Site Settings

This Site Settings overview page will show the SFTP details and allow you to toggle the site between inactive and active. These details are preset by Omeda and cannot be edited.

For third-party SFTP sites, you will have the same visibility once the site has been successfully set up.

For the Omeda SFTP site, the file path you can use to locate/deposit files will be visible in the ‘Host’ spot:

Email Lists

Email Lists will allow you to set up email addresses to receive relevant Data Loader FTP updates.

Clicking ‘Add Email Address’ will generate the following popup:

Once you have added an email address as a recipient, you can choose which type(s) of emails they will receive as well as which files they’ll receive the notifications for.

  • All: Automatically checks all of the email lists and will add the email address to all of them.

  • Overview (Checked by Default): This will send 1 (one) email every time the FTP Upload job runs. It will include stats on how the job ran. This is a brief overview of what happened; opt into other mailing lists for more detailed information, such as the “Needs Mapping” or “Errors” mailing lists.

  • Processed: This will send 1 (one) email per File that has been successfully processed during an FTP Upload job run. The resulting file will likely be in the “Completed” or “Completed with Errors” state. This is the same email that is received when a File is processed manually. If multiple files process automatically, the email address(es) will receive one email per file.

  • Needs Mapping: This will send 1 (one) email every time the FTP Upload job runs. It contains up to two different sets of information:

    • File Matched Criteria, but not Template – This will list out the files that had issues matching the Template assigned to the Tracking criteria the file’s name matched on. It will then list the items that caused it not to match the Template. This can be either mismatching headers (extra or missing) or values that have not been mapped. In any case, the File will be uploaded and the user can correct the Mapping in the Files Section, as they would if it were a manually uploaded file.

    • File didn’t Match Tracking Criteria – This will list files that existed on the FTP Site but could not be properly matched to a Tracking criteria. No actions were taken upon these files.

  • Errors: If the FTP Upload job fails, 1 (one) email will be sent informing the assigned address that something has gone wrong with the upload job. This will likely be due an issue with how the FTP Site or Tracking criteria were setup.
    Additionally, this mailing list will potentially send 1 (one) email per File that fails during the Upload or Processing steps. These will be the standard emails that are sent normally when a File fails in either situation.

  • File Name: Select which file (from Manage Tracking) the recipient should receive notifications for. Otherwise, keep it defaulted to ‘All’ to receive notifications for all files uploaded to the FTP site. If recipient should receive notifications for more than one file, but not all, add the Email Address to list again.

Manage Tracking

To set up how files are pulled from the FTP site and ingested into your database, you’ll need to go into ‘Manage Tracking’. Files are identified and matched to templates based on tracking rules.

To set up a new tracking rule, click ‘Add File Name to Track’. This will open up a new popup where you will need to 1) Enter a file name 2) Note how the file is delimited, and 3) Select a saved template to apply to each of the files that fall into the track.

For more information on how to set up a template for Data Loader, see our Templates article.

Once the track is set up, it will appear in the Manage Tracking list view. Multiple tracks can be set up and active at the same time. If multiple tracks match a file, the highest priority will be applied.

From the list view you are easily able to edit the file name, change the delimiter, adjust the match criteria, change the template and update the priority.

From the actions drop-down you can also remove and activate/inactivate tracks.

View Directory

Within the ‘View Directory’ section, you have the ability to see all of the files currently on the FTP site and if they have been matched and their status.

The file statuses that you will see are:

  • Available: The file can be processed if there is a matched tracking assigned. If none is assigned, you can set one up using the ‘Track File’ option under Actions.

  • Uploaded: The file was pulled in from the FTP site, but couldn’t be processed (usually because the template didn’t match the file 100%). You will need to review the mapping and manually process the file via the standard Data Loader process.

  • Processed: The file was successfully pulled in, the template was applied, and the data was added to your database.

  • Failed: The file was not processed, or uploaded.

 

Any additional questions related to the FTP processing or Data Loader in general, please contact your Audience Services team.