Optimizing for-each loops in Logic Apps with strings

I came to a challenging scenario where I needed to extract data from a cloud system by making several calls to that system API’s and synchronize that with another system. The main issue was not the API’s calls themselves as I found a way to optimize those requests but how to handle the response from those calls as I needed to filter some records for further processing.

So, for every response from that API, I needed to filter only records that meet a certain criteria and add them to a list to be returned for the caller of the Logic App.

This is where I started having several problems, as we know that the for-each loops in Logic Apps executes the code in paralel, so our code needs to consider that when handling variables.

Problem

In the picture below you can see part of the implementation that I did where I make the API call that is querying about 500 records. The call returns in 8 seconds, which is an OK time, and since I’ll be doing maybe 10 calls of this API each with 500 records, having the response for 5,000 through API’s in 8 seconds is fairly good (remember that they will run inside a for-each so will run in parallel).

The problem as you can see in the picture is that the for-each loop to add each item to an array variable (it’s not possible to add the full response from the filter to the array, I tried), is taking 36 seconds for this part of the for-each execution.

If this was the only filter and handling that I was going to do it was not that bad, but I had to this a few other times, which would keep adding more and more time to my data synchronization process, so this solution was not feasible with this usage of for-each.

Solution

As a developer I try to avoid the use of string concatenation as much as possible, but in this case as you can see in the picture below, that was the best option since the performance of the append to array variable action was very poor.

The Append to actions are only concatenating the response from the API into a string variable and in the Set actions, I’m executing the following code to make that string an usable JSON that I’ll be handling later.

As you can, the performance difference is massive and my code is also reduced to a minimum set of actions.

After I applied this logic in the other parts of my Logic App, the execution time dropped from 10 minutes to about 25 seconds.

Note: Logic Apps Monitoring is done proactively on multiple metrics and properties in a single alert at no additional charge and set threshold limits.

Summary

In this post I show how to use string concatenation to obtain a better performance from Logic Apps when using for-each loops instead of arrays.

Have a read and let me know what do you think about this implementation.

Cheers!!!