Archive

Posts Tagged ‘ExecuteMultipleRequest’

[Code Snippet] Using Microsoft.Pfe.Xrm library

September 20, 2020 Leave a comment

Pfe.Xrm library allows us to submit a bunch of XRM requests as batches (using ExecuteMultipleRequest) and execute them in parallel.

In this article, lets see how to use Microsoft.Pfe.Xrm library with a simple ‘Retrieve Request’.

What is Microsoft.Pfe.Xrm library?

  • Contains a set of common components for building solutions using the Dynamics 365 SDK and is developed by Microsoft Premier Field Engineering (PFE) and core engineering teams.

Download Microsoft.Pfe.Xrm

  • Use ‘Tools -> Nuget package manager’ from your Visual Studio to download the Nuget package.

Building blocks of Microsoft.Pfe.Xrm library

‘Microsoft.Pfe.Library’ allows us to trigger any ‘OrganizationRequest’ in batches (i.e.,ExecuteMultipleRequest) parallelly. Lets understand the building blocks of Pfe library.

  • List<OrganizationRequest>
    • As a first step prepare your ‘RetrieveMultipleRequest’ objects and add them to a List.
  • AsBatches() method
    • Call Pfe libraries AsBatches() method by passing the List prepared in previous step.
    • ‘AsBatches()’ returns IDictionary<string, ExecuteMultipleRequest> object.
  • OrganizationServiceManager
    • Trigger ‘Microsoft.Pfe.Xrm’ OrganizationServiceManager() method to create connection object.
    • There are multiple ‘OrganizationServiceManager’ overload methods.
    • In this article, I am using the following method.
  • ParallelProxy.Execute
    • This is the final method which accepts the ‘IDictionary<string, ExecuteMultipleRequest>’ object returned by AsBatches() method which we discussed in previous sections.
    • ParallelProxy.Execute also requires OrganizationServiceManager which we created in previous step.

IDictionary<string, ExecuteMultipleResponse> batchResponses = pfeXrmConnection.ParallelProxy.Execute(batchesOrgRequest);

Code Snippet

Now lets put all the building blocks together. It becomes as the following code snippet.

    public void RetrieveUsingPFE()
    {
        // Retrieve 'Account' and 'Contact'. You can prepare your list of entities.
        var listEntities=new List<>{ "account", "contact" };
        var batchRequests = new List<OrganizationRequest>();            

        int pageNumber = 1;
        string pagingCookie = null;

        // Filter for Active records. You can prepare your required filters as below.
        var filterActive = new FilterExpression(LogicalOperator.And);
        var condActive = new ConditionExpression("statecode", ConditionOperator.Equal, 0);
        filterActive.AddCondition(condActive);

        try
        {
            foreach (var entity in listEntities)
            {
                // Prepare 'Query Expression'
                var query = new QueryExpression(entity)
                {
                    ColumnSet = new ColumnSet(true),
                    PageInfo = new PagingInfo()
                };
                query.PageInfo.Count = 5000;
                query.PageInfo.PageNumber = pageNumber;
                query.PageInfo.PagingCookie = pagingCookie;

                query.Criteria.AddFilter(filterActive);

                var request = new RetrieveMultipleRequest
                {
                    Query = query
                };

                batchRequests.Add(request);
            }

            do
            {
                // Trigger Pfe.Execute
                var batchResponses = TriggerRequests(batchRequests);
                batchRequests = new List<OrganizationRequest>();
                pageNumber++;
                foreach (var responseItem in batchResponses)
                {
                    if (responseItem.EntityCollection.Entities.Count > 0)
                    {                            
                        var currEntityName = responseItem.EntityCollection.Entities[0].LogicalName;

                        foreach (var record in responseItem.EntityCollection.Entities)
                        {
                            // 'record' is the Retreieved record
                            // Perform your desired operations
                        }
                    }

                    if (responseItem.EntityCollection.MoreRecords)
                    {
                        var query = new QueryExpression(responseItem.EntityCollection.EntityName)
                        {
                            ColumnSet = new ColumnSet(true)
                        };

                        query.Criteria.AddFilter(filterActive);

                        var request = new RetrieveMultipleRequest();
                        query.PageInfo = new PagingInfo
                        {
                            Count = 5000,
                            PageNumber = pageNumber,
                            PagingCookie = responseItem.EntityCollection.PagingCookie
                        };

                        request.Query = query;
                        batchRequests.Add(request);
                    }
                }
            } while (batchRequests.Count > 0);
        }
        catch (Exception ex)
        {
            Console.WriteLine("Error occurred, error message is {0}", ex.Message);
            throw ex;
        }            
    }

    public List<RetrieveMultipleResponse> TriggerRequests(List<OrganizationRequest> batchRequests)
    {
        var retrieveRequestBatches = batchRequests.AsBatches(recordsPerRequest);
        var batchResponses = ExecuteParallelProxy(batchRequests);            
        var listResponses = new List<RetrieveMultipleResponse>();

        foreach (var key in batchResponses.Keys)
        {
            foreach (var result in batchResponses[key].Responses)
            {
                if (retrieveRequestBatches[key].Requests[result.RequestIndex] is RetrieveMultipleRequest)
                {
                    var originalRequest = (RetrieveMultipleRequest)retrieveRequestBatches[key].Requests[result.RequestIndex];
                    // Capture failed records
                    if (result.Fault != null)
                    {
                        Console.WriteLine($" Exception : {result.Fault.Message}");
                    }
                    else if (result.Response != null && result.Response is RetrieveMultipleResponse) // Capture success records
                    {
                        var responseItem = (RetrieveMultipleResponse)result.Response;
                        listResponses.Add(responseItem);
                    }
                }                    
            }
        }

        return listResponses;
    }

    private IDictionary<string, ExecuteMultipleResponse> ExecuteParallelProxy(IDictionary<string, ExecuteMultipleRequest> retrieveRequestBatches)
    {            
        IDictionary<string, ExecuteMultipleResponse> batchResponses = null;
        try
        {
            batchResponses = CRMPfeConnection.ParallelProxy.Execute<ExecuteMultipleRequest, ExecuteMultipleResponse>(retrieveRequestBatches);                
        }
        catch (AggregateException ae)
        {
            foreach (var ex in ae.InnerExceptions)
            {
                Console.WriteLine($"Aggregate Error in ExecuteParallelProxy : {ex.Message.ToString()}");
            }
        }
        catch (Exception ex)
        {
            Console.WriteLine($"Error in ExecuteParallelProxy : {ex.Message.ToString()}");
        }

        return batchResponses;
    }

    public static OrganizationServiceManager CRMPfeConnection
    {
        get
        {
            if (crmPfeConnection == null)
            {
                var UserName = ConfigurationManager.AppSettings["UserNameSrc"].ToString();
                var Password = ConfigurationManager.AppSettings["PasswordSrc"].ToString();
                var OrganizationUri = ConfigurationManager.AppSettings["OrganizationUriSrc"].ToString();
                ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls11 | SecurityProtocolType.Tls12;
          crmPfeConnection = new OrganizationServiceManager(XrmServiceUriFactory.CreateOrganizationServiceUri(OrganizationUri), UserName, Password);
            }
            return crmPfeConnection;
        }
    }
  • Similar to ‘RetrieveRequest’, you can trigger all kinds of requests.

🙂

Transaction aware batch processing using ‘ExecuteMultipleRequest’

April 10, 2018 4 comments

In this article, I am going to discuss about transaction aware batch processing in ExecuteMultipleRequest using  ExecuteTransactionRequest.

Is ‘ExecuteMultipleRequest’ not a transaction aware?

  • By design, ‘ExecuteMultipleRequest’ is non-transactional.
  • Which means ‘Request’ objects added to ‘ExecuteMultipleRequest’ are independent of each other and if any exception occurred in any of the ‘Request’ will not roll back previously committed ‘Requests’.
  • However you have an option to stop the batch (i.e., By setting ContinueOnError = false),  if any exception occurs.
  • Important to note is that, this behavior is not a transaction aware and ‘Requests’ executed prior to the error, will still be committed.

How can we achieve ‘Transaction’ in Batch execution?

  • Using ExecuteTransactionRequest, we can achieve ‘Transaction’ (i.e., Commit and Rollback) in Batch execution
  • ExecuteTransactionRequest will execute the all the ‘Requests’ in a single transaction.
  • If any exception occurred in any of the ‘Request’ execution, will roll back all the previously executed ‘Requests’.

Then why do we need ‘Transaction aware’ feature in ‘ExecuteMultipleRequest’?

  • ‘ExecuteMultipleRequest’ has an option to either stop the operation or suppress the exception, if any exception occurs, using ContinueOnError property.
  • However there is no option to achieve ‘Transactional Batch feature’ in ExecuteMultipleRequest.

Lets see how we get both features together in ‘ExecuteMultipleRequest’?

[Code snippet] ‘ExecuteMultipleRequest’ with ‘Transaction aware’ option

  • ‘ExecuteMultipleRequest’ can have ‘ExecuteTransactionRequest’ object as a ‘Request’, and as we know ‘ExecuteTransactionRequest’ is transaction aware, it get executed in transaction.

var connectionString = ConfigurationManager.ConnectionStrings[“Xrm”].ConnectionString;
CrmServiceClient crmConn = new CrmServiceClient(connectionString);

// Instantiate ExecuteMultiple
var reqExecMultiple = new ExecuteMultipleRequest(){
Requests = new OrganizationRequestCollection(),
Settings = new ExecuteMultipleSettings(){
ContinueOnError = false,
ReturnResponses = true
}
};

// Instantiate ExecuteTransaction
var reqExecTransaction = new ExecuteTransactionRequest(){
Requests = new OrganizationRequestCollection()
};

// Add 20 requests of type ‘CreateRequest’ to ‘Execute Transaction’
for (int indxStudent = 0; indxStudent < 20; indxStudent++){
// Instantiate ‘Student’ record
var entStudent = new Entity(“new_student”);
entStudent[“new_name”] = “Student : ” + indxStudent.ToString();

// Instantiate ‘CreateRequest’
CreateRequest reqCreate = new CreateRequest{
Target = entStudent
};

// Add ‘CreateRequest’ object to ‘ExecuteTransaction’ Requests collection
reqExecTransaction.Requests.Add(reqCreate);
}

// Execute the ‘Execute Multiple Request’
using (_serviceProxy = crmConn.OrganizationServiceProxy){
_service = _serviceProxy;

// Add ‘Execute Transaction’ request object to ‘ExecuteMulitple’ Requests collection
reqExecMultiple.Requests.Add(reqExecTransaction);

// Execute Multiple Request
var response = (ExecuteMultipleResponse)_service.Execute(reqExecMultiple);

// Read the GUID of created Students
foreach (var responseItem in response.Responses){
var execTransactionResponse = (ExecuteTransactionResponse)responseItem.Response;
foreach (var createResponse in execTransactionResponse.Responses){
Console.WriteLine(“Student created: {0}”, ((CreateResponse)createResponse).id);
}
}
}

ExecMultiple

Key Notes:

  • ‘ExecuteTransactionRequest’ cannot contain either ‘ExecuteTransactionRequest’ or ‘ExecuteMultipleRequest’ messages.
  • Its recommended to not use ‘ExecuteTransactionRequest’ or ‘ExecuteMultipleRequest’ in Plug-ins. Refer article.
  • Operations included in a batch request are executed sequentially and aren’t done in parallel.
  • ‘ExecuteTransactionRequest’ may potentially block the database for the duration of the long-running transaction, so use it judiciously.

🙂