A way to process more than 5000 records in Cloud Flow execution

If you need to process a large number of records, for example in Dataverse, what we really do is create a scheduled Cloud Flow/ Power Automate. If the process is complex, we can encapsulate the logic in a Custom API and call from Cloud Flow. 

To fetch the records to process, you need to use List Rows operation of Dataverse Connector which returns only 5000 records as once. This is a limitation. In order to overcome this, we need to do below trick.

1) Introduce a Boolean Variable

2) Create a Do Until loop to run until this variable is False

3) Inside the Loop

    - Create List Rows operation to retrieve records

    - Set Variable to True as far as List Rows returns none zero records

4) Add a For Each control to call a Bound Operation against each record in the batch.

Caution

1) List Rows fetch should contains a specific condition for the records being fetched. (perhaps a flag)

2) Within the Custom API it is important to change this flag value once its processed either its successful or failure. 

This way, any record, once processed is omitted in next fetch resulting only one processing per a given record. Otherwise, this will go to an infinite loop which could cause issues. 

Anyway, Cloud flows can handle badly designed loops and it will anyway stop at default maximum number of loops.. still this should be avoided.

Attach PDF file programmatically to Email and send

Let's see how PDF file is attached to an Email and sending programmatically. Lets see why this is important.

1) Email Templates cannot be created with an attachment. (Ironically!)
2) If we attach programmatically we can change the attachment based on logic if required.

In Summery, we can initiate Email in Draft state, then attach the PDF and then send.

Initiation and sending the e-mail part is explained in Programmatically create a draft Email using Email Template (C#).

Lets see the code

// Initiate Email in Draft state

// Create Attachment
byte[] embededPdf = LoadEmbeddedPdf("PluginProject.Resources.ApplicationForm.pdf")
Guid attachmentId = CreatePdfAttchment(OrgSvc,emailId, Convert.ToBase64String(embededPdf));

// Send Email

// Methods to use
private byte[] LoadEmbeddedPdf(string resourceName)
{
    var assembly = Assembly.GetExecutingAssembly(); 
    using (Stream stream = assembly.GetManifestResourceStream(resourceName))
    {
        if (stream == null)
            throw new FileNotFoundException("Embedded PDF file not found: " + resourceName);
        using (MemoryStream ms = new MemoryStream())
        {
            stream.CopyTo(ms);
            return ms.ToArray();
        }
    }
}

public Guid CreatePdfAttchment(IOrganizationService OrgSvc, Guid emailId, string base64Pdf)
{
    Entity attachment = new Entity("activitymimeattachment");
    attachment["subject"] = "Application Form";
    attachment["filename"] = "ApplicationForm.pdf";
    attachment["mimetype"] = "application/pdf";
    attachment["body"] = base64Pdf;
    attachment["attachmentnumber"] = 1;
    attachment["objectid"] = new EntityReference("email", emailId);
    attachment["objecttypecode"] = "email";
    return OrgSvc.Create(attachment);
}

There are two important things here;

1. PDF is saved within the Project and path needs to be passed correctly in below standard.

<Project Namespace><Folder><filename>.pdf

So you will realize when I pass PluginProject.Resources.ApplicationForm.pdf, my project namespace is  PluginProject, folder name is Resources and pdf file name is ApplicationForm.pdf.

2. Go to properties of the PDF file (In VS) and do below setting.