Efficient Data Queries & Pagination for SharePoint & Dataverse with Power Automate

In this video, I look at why the default "Get Items" action for SharePoint and "List Rows" action for Dataverse might not be the most efficient choice due to the excessive metadata they return. Learn through how to bypass this by directly querying these data sources using HTTP requests, significantly improving performance by reducing unnecessary data overhead.
00:00 Introduction: Why you shouldn't use the Get Items or List Rows action.
01:16 Comparing Performance of Get Items action for SharePoint to a HTTP Request.
05:55 Comparing Performance of List Rows action for Dataverse with HTTP.
08:26 Using pagination with the HTTP Method.
11:33 Comparison of Performance when using pagination.
Working with lists and list items:
learn.microsoft.com/en-us/sha...
Working with Parse JSON:
• Unlock the Full Potent...
Further reading JSON Light Support in SharePoint REST API
www.microsoft.com/en-us/micro...
If you found this video useful, please consider making a donation to the channel:
www.buymeacoffee.com/paulie

[@DavidAdediran]

Great video Paul. The Dataverse's list rows action does support the selection of columns, so you can achieve similar performance with the added benefit of better Power Automate support (parsing, auto-suggestion, etc.)

[@PaulieM]

It does, but it still returns a stack of extra metadata. But the column selection certainly helps.

[@lubeshbehera1993]

Thanks, Paul, for these wonderful suggestions!!

[@skim32]

Does using the Send an HTTP request to SharePoint also provide better loop internation performance compared to the get items? Or is it only when getting the data? For example. Performing a Apply To Each for each record.

[@kimsalas8197]

Thanks for the video, Paul. Would the opposite of your method work if you did a POST method in order to write thousands of records to Dataverse? Do you have a video for that?

[@PaulieM]

Yes, it is absolutely possible. But not in the same way as shown in this video. You would need to use the $batch API method. I have a video on batch deleting records from Dataverse. I could make one on batch insert if you think it would be useful.

[@kimsalas8197]

@@PaulieM Yes, that would be great if you could make one on batch insert! Thank you!

[@michaelseerup]

Great video, is it possible to achieve similar improvements directly from Power Apps?

[@MedariKeshava]

Hi paul, Thanks for the video!
I have a question, is there a way to load more than 20,000 records from sharepoint in to power apps, with power automate, using only powerfx formulas with search querying all data.

[@ukm365]

I think the other thing that the 'Actions' are doing is that they're 'streaming' the data. This enables more connective data from lookup fields and suchlike.
HTTP calls very rarely include much lookup information unless you specifically request it, right?

[@PaulieM]

I think the Actions are doing much the same thing as the HTTP, but with the odata set to verbose. You can even see the setting in the HTTP header of the outputs. But I cannot be sure of course.