Skip to main content
R
RonnieKarpinsk
Newcomer
December 11, 2021
Solved

Batch Automation Uploads

  • December 11, 2021
  • 4 replies
  • 0 views

I am looking for tips on batch automation business rules. Currently when our scheduled trial balances are loaded the whole upload will fail if one transformation mapping error is present. As you can imagine this can slow the close process down significantly. Does anyone have any suggestions on how to allow the batch to load the validated accounts and only error out the transformation errors? 

Best answer by tsandi

You can create a mask for unmapped values across every dimension and use the Order to make sure that will be the last rule applied, but it is a VERY BAD IDEA since the whole purpose of a data quality layer is to make sure the whole dataset gets loaded. If your customer has data quality issues and you want to use OneStream to analyze that data you can use a Blend load with Garbage mask rules, allows the users to review the transactions in blend and create a second load with the updated mappings. 

tsandi_0-1639413092489.png

If the source GL data has additional metadata that needs to be created you can include the metadata creation in the connector rule itself so loads will find the appropriate members when loading data. 

 

4 replies

L
LeeB
Contributor
December 11, 2021

There isn't a "load with errors" option in OneStream.

I've seen solutions for overnight loads where unmapped values are caught in a mask transformation rule which maps them to a member named "Unmapped" or similar.  An Administrator would then review and remap any accounts mapped to this member.

However, automatically importing incomplete data sets such as Trial Balances presents some obvious issues.  Where possible, I would favour implementing some form of master data governance to ensure any changes are identified and dealt with across all systems thereby preventing the mapping errors from occuring.

    T
    OneStream Employee
    tsandiOneStream EmployeeAnswer
    OneStream Employee
    December 13, 2021

    You can create a mask for unmapped values across every dimension and use the Order to make sure that will be the last rule applied, but it is a VERY BAD IDEA since the whole purpose of a data quality layer is to make sure the whole dataset gets loaded. If your customer has data quality issues and you want to use OneStream to analyze that data you can use a Blend load with Garbage mask rules, allows the users to review the transactions in blend and create a second load with the updated mappings. 

    tsandi_0-1639413092489.png

    If the source GL data has additional metadata that needs to be created you can include the metadata creation in the connector rule itself so loads will find the appropriate members when loading data. 

     

      G
      Gidon_Albert
      Contributor
      January 3, 2022

      We use the TransfromationEventHandler to either add new members under "Unmapped_Members" parent if the we are using one-to-one mapping, or map them to an "Unmapped_Members" base level member if we are using many-to-one mapping. The rule captures the name of the new members that are added and sends an e-mail to the admin to let them know that new members were added. Admin still has to move the new members or update the maps as needed, but at least the load process doesn't fail.

        A
        Ando
        Newcomer
        October 5, 2022

        Would you have an example of that rule you are using to do that?

        G
        Gidon_Albert
        Contributor
        October 5, 2022

        The rule we use to dynamically update members is highly customized and specific to our application. It will require some work to make it work for your needs.

        OneStream has a MarketPlace solution called ACM that can update new members during data load. It works well and is easy to configure. It is also free!

        Let me know if the ACM doesn't work for you and I'll be happy to share our TransformationEventHandler.

        Terms & ConditionsAccessibility statement