Advertisements

Archive

Archive for March, 2018

C# – SQL – Bulk Insert records from Data Table and CSV

In my previous article I provided the steps to update records in single transaction using SQL “Table-Valued Parameter Types”.

In this article, lets see an easy way to bulk insert the records using SQLBulkCopy

What is SQLBulkCopy:

  • SQLBulkCopy class comes with ‘System.Data’ namespace and can be used to write/Insert data to SQL Server tables.
  • Though there are other ways to load data into a SQL Server table (i.e., INSERT statements, for example) but ‘SQLBulkCopy’ offers a significant performance advantage.

Below are the steps to use SQLBulkCopy with the combination of ‘Data Table’

Steps:

To simplify the explanation, I am taking a ‘Student’ table with 3 columns.

  • Create a ‘Student’ table.

Data Table 1

  • In C# console application, prepare a ‘Data Table’ with the structure similar to ‘Student’ SQL table.

tableStudents = new DataTable(“Student”);
tableStudents.Columns.Add(“StudentID”, typeof(string));
tableStudents.Columns.Add(“StudentName”, typeof(string));
tableStudents.Columns.Add(“City”, typeof(string));

// Add ‘Student’ data as Rows to the Data Table
DataRow rowStudent = tableStudents.NewRow();
rowStudent[“StudentID”] = “RJ001”
rowStudent[“StudentName”] = “Rajeev”;
rowStudent[“City”] = “Hyd”;

//Add the Data Row to Table
tableStudents.Rows.Add(rowStudent);

  • Instantiate and execute the ‘SQLBulkCopy’ by passing the Data Table

using (SqlBulkCopy bulkCopy = new SqlBulkCopy({SQLConnection}, SqlBulkCopyOptions.TableLock | SqlBulkCopyOptions.FireTriggers | SqlBulkCopyOptions.UseInternalTransaction, null))
{
bulkCopy.DestinationTableName = “Student“; // Your SQL Table name

//Optional mappings. Not required, if your ‘Data Table’ column names match with ‘SQL Table’ column names
bulkCopy.ColumnMappings.Add(“StudentID”, “StudentID”);
bulkCopy.ColumnMappings.Add(“StudentName”, “StudentName”);
bulkCopy.ColumnMappings.Add(“City”, “City”);
bulkCopy.WriteToServer(tableStudents);
isSuccuss = true;
}

  • Execute the code which inserts rows from ‘Data Table’ to ‘Student’ SQL table.

Now lets see how to insert the CSV data to SQL table with query.

Note: You can also do the same with SQL Import Wizard

Bulk insert records from CSV to SQL Table:

  • Assume that you have data available in a CSV file and want to insert them as records in SQL Table.

Insert 1

  • Below is the query

Bulk Insert {SQL Table Name}

from ‘CSV File Path’

with
(rowterminator=’\n’,fieldterminator=’,’)

Insert 2

Advertisements
Categories: SQL Tags: , ,

C# – Bulk Update Records – Pass Data Table to SQL Stored Procedure

March 30, 2018 3 comments

I was building a console application to transact with large set of data in a SQL table and got a requirement to bulk update records.

Below are the options I got.

Option 1:

  • Loop through the records in Console and make SQL update call for each record.
  • This is a tedious and non-scalable for huge volumes data, as we need to make one SQL server transaction for each record from client (i.e, Console).

Option 2:

  • Populate the records to update as rows in a ‘Data Table’ and pass it to a Stored Procedure, and have the Update statement in Stored Procedure.
  • This is optimal as the heavy  SQL updates are off loaded from Console to SQL server.

Lets see how to implement options 2.

To make the Stored Procedure accept Data Table, we need to define “Table-Valued Parameter Types” in SQL first.

What is a ‘Table-Valued Parameter Type”:

  • Table-valued parameters are based on strongly-typed table structures that are defined by using SQL CREATE TYPE statements.
  • You need create a “Table-Valued Type” (i.e.,StudentTableType from below scenario) with the structure similar to your Table (i.e., Student table from below scenario).

Steps to implement Option 2:

To simplify the explanation, I am taking a ‘Student’ table with 3 columns.

  • Create ‘Student’ Table

Data Table 1

  • Create a ‘User-Defined Table Type’ (i.e., ‘StudentTableType‘) with ‘Student’ columns.
    • Note: You don’t need to include all columns. Add columns only you need to update.

Data Table 2

  • Create a Stored Procedure with StudentTableType as Parameter

Data Table 3

  • In the Console, populate a ‘Data Table’ with ‘Student’ records as ‘Data Row’

tableStudents = new DataTable(“Student”);
tableStudents.Columns.Add(“StudentID”, typeof(string));
tableStudents.Columns.Add(“StudentName”, typeof(string));
tableStudents.Columns.Add(“City”, typeof(string));

// Add ‘Student’ data as Rows to the Data Table
DataRow rowStudent = tableStudents.NewRow();
rowStudent[“StudentID”] = “RJ001”
rowStudent[“StudentName”] = “Rajeev”;
rowStudent[“City”] = “Hyd”;

//Add the Data Row to Table

tableStudents.Rows.Add(rowStudent);

  • Execute the Stored Procedure from Console by Passing ‘DataTable’ (i.e.,tableStudents) as Parameter

using (SqlConnection connection = ConnectionManager.SQLConnection)
{
SqlCommand cmdUpdateStudents = new SqlCommand(“BulkUpdateStudents”, connection)
{
CommandType = CommandType.StoredProcedure
};

SqlParameter tvpParam = cmdUpdateStudents.Parameters.AddWithValue(“@tvpStudents”, tableStudents);
tvpParam.SqlDbType = SqlDbType.Structured;

cmdUpdateStudents.ExecuteNonQuery();
}

 

 

 

Dynamics 365 – Configure ‘Relevance Search’

In Dynamics 365 online we have 2 types of Global Search

  • Category Search
  • Relevance Search

Rel Search_8

Category Search:

  • Its a multi-entity Quick Find Search, where you can pull the matching records from all configured entities in a single screen.

Relevance Search:

  • Relevance Search delivers fast and comprehensive results across multiple entities in a single list, sorted by relevance.
  • It uses a dedicated search service external to Dynamics 365 (powered by Azure) to boost Dynamics 365 search performance.

Lets see how to configure ‘Relevance Search’ in Dynamics instance and enable an entity.

Enable Relevance Search:

  • You have to enable Relevance Search first, Go to Settings -> Administration -> Check the ‘Enable Relevance Search’ checkbox

Rel Search_1

Set Entities Managed Property:

  • To enable entity for ‘Relevance Search’, make sure Managed property set to ‘True’. This property will be ‘True’ by default.

Rel Search_2

Add Entity to the list:

  • Open the ‘Default Solution’ -> Configure Relevance Search
  • Add the entity to the list

Rel Search_3

Set default global search:

  • You can choose the default search between ‘Categorized Search’ and ‘Relevance Search’ from your ‘Personal Settings’.
  • Last search option provides the last search experience you used in the current browser client. For example, if you used Relevance Search for your last search, the system will continue using this search until you manually switch to Categorized Search.

Rel Search_6

Notes:

  • ‘Relevance Search’ is only available in Dynamics 365 online.
  • ‘Relevance Search’ is text-based, and can search only on fields of type Single Line of Text, Multiple Lines of Text, Option Sets, or Lookups. It doesn’t support searching in fields of Numeric or Date data type.
  • You can use syntax to search with combinations. For example, type car+silver+2-door to find only matches that include all three words. Type car|silver|2-door to get results that contain car or silver or 2-door, or all three words
  • There is no limit on no of entities can be included in the Relevance Search results. However, there is a limit on the total number of fields in the external search index. Currently, the maximum is 1000 searchable fields for an organization.

🙂

SSIS – Read C# ‘List’ collection in ‘Execute SQL Task’

In one of our integration requirements, we had to read the ‘List<>’ collection values, which was set in ‘Script Task’ control and pass to ‘Execute SQL Task’ control.

In this article I am going to list out the step by step process to set and get the ‘List<>’.

Assume that you have a collection List<string> which holds ‘Product Names’

var listProducts  = new List<string>() {“Product1″, ” Product2″, ” Product3″};

Lets see how to loop through and read the ‘Product names’ and pass to ‘Execute SQL Task’.

Required SSIS components:

  • A SSIS ‘Variable’ (i.e., varProducts) of type ‘Object’ to set the ‘List<>’ object in ‘Script Task’ control

SSIS_1

  • A SSIS ‘Variable’ (i.e., varProductName) of type ‘String’ to set the ‘Product Name’ for each List<> value.

SSIS_2

  • ‘Foreach Loop container’ to parse and read the ‘List<>’

Steps:

  • Choose ‘varProducts’ as ‘ReadWriteVariables’

SSIS_3

  • In ‘ScriptTask’ set ‘listProduct’ object to ‘varProducts’ variable.
    • var listProducts  = new List<string>() {“Product1″, ” Product2″, ” Product3″};
    • Dts.Variables[“User:: varProducts”].Value = listProducts ;
  • In ‘Foreach Loop container’,
    • Choose ‘Enumerator’ as ‘Foreach From Variable Enumerator’
    • Enumerator Variable as ‘User:varProducts’

SSIS_4

  • As the ‘List’ contain ‘Product Name’ collection, read the ‘Product Name’ to ‘varProductName’ variable. ‘Index’ 0 denotes, first value in collection (i.e., Product Name’.

SSIS_5

  • As the final step, In ‘Execute SQL Task’, map the ‘varProductName’ variable to a ‘Parameter’.

SSIS_6

  • In above screen, we are setting ‘@prodName’ parameter with ‘Product Name’ and ‘@prodName’ further used in ‘SQL Statement’ of ‘Execute SQL Task.

🙂

Categories: SQL Tags: ,

[Step By Step] Configure Server-to-Server (S2S) authentication using Azure AD and Application User – Dynamics 365

In this article I am going to explain, what is ‘Application User’ and how it helps to establish using Server-to-Server (S2S) authentication and Azure Active Directory

To explain the S2S authentication simpler, let’s take an integration requirement

  • You have an ASP.Net Web Application
  • You need pull the Contacts from a CRM organization and display in the ASP.Net Web Page

The conventional design approach for the above requirement would be

  • Establish the CRM connection in your ASP.Net page by passing CRM User credentials
  • Make a Retrieve call to CRM
  • Read and bind the Contacts to a grid.

To implement the above design you need to have a paid CRM User credentials to interact with your Dynamics CRM organization.

So what is S2S authentication and how is it different from the legacy integration model we discussed above.

Server-to-Server (S2S) authentication:

  • S2S authentication means you don’t need to use a paid Dynamics 365 user license when you connect to Dynamics 365 tenants.
  • We will use a special user (i.e., Application User)
  • Best part is, you can connect to D365 and make server calls from your application (i.e.,Web/Console) with no Dynamics SDK dlls and no ‘UserID/Password’.

What is this ‘Application User’:

  • ‘Application User’ is a ‘systemuser’ record of type ‘Application User’
  • There is no license fee for the ‘Application User’ account

App User - 14

How an ‘Application User’ account achieve the S2S authentication:

  •  ‘Application User’ with conjunction of Azure Active Directory (Azure AD) will establish S2S authentication.
  • We first generates an ‘Application ID’ in Azure AD which we set in ‘Application User’ in Dynamics.

Lets see the step by step approach to achieve S2S authentication.

  • Pre-requisites:
    • Dynamics 365 instance
    • Azure Subscription with same Office 365 account used for your D365 instance.
  • High Level Steps
    • Generate ‘Application ID’ and ‘Keys’ in ‘Azure’
    • Add a new User in ‘Azure Active Directory’ (Azure AD)
    • Create a new ‘Application User’ in Dynamics 365

Step 1 – Generate ‘Application ID’ and ‘Keys’ in ‘Azure’:

  • Connect to your Azure
  • Go to ‘App registrations’ service

App User - 1

  • Create a ‘New application registration’
    • Note: ‘Sign-on URL’ can be any valid URL.

App User - 2

  • Copy the generated ‘Application ID’ (This is needed while creating ‘Application User’ in CRM)

App User - 3

  • Generate ‘Keys’ (You need the ‘Key’ to establish connection in your Web Application/Console Application)

App User - 4

  • Save the ‘Key’ (Note: You cannot read the key if you move away from the screen)

App User - 5

Step 2 – Add a new User in ‘Azure Active Directory’ (Azure AD):

  • Connect to your Azure
  • Go to ‘Users’ service

App User - 6

  • Create a ‘New User’
    • Note: ‘Password’ auto generates once you save. You don’t need to copy as this is not required further.

App User - 7

  • Once the User saved, copy the ‘User Name’ (This is needed while creating ‘Application User’ in CRM)

App User - 8

Step 3 – Create a new ‘Application User’ in Dynamics 365:

This step we are going to create an ‘Application User’ in D365 by copying the details generated in Azure

  • Connect to Dynamics 365
  • Go to ‘Settings -> Security -> Users
  • Switch the view to ‘Application Users’ and click ‘New’

App User - 9

  • In the ‘New User’ screen
    • Set ‘User Name’ with the ‘User Name’ copied from ‘Azure’
    • Set ‘Application ID’ with the ‘Application ID’ copied from ‘Azure’
    • Save the User and once saved, you notice the auto populated ‘Application ID URI’ and ‘Azure AD Object ID’

App User - 10

  • Assign a ‘Security Role’
    • ‘Security Role’ must be a Custom Security Role and you cannot assign OOB role.
    • For this exercise, you might want to copy any existing OOB Security Role.

All right! We are all set and now its time to test S2S authentication from your console.

S2S Authentication Code Snippet:

Prerequisites:

  • Install ‘ADAL’ and ‘NewtonSoft’ NuGet packages

 Code:

using Microsoft.IdentityModel.Clients.ActiveDirectory;
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
using System;
private static async Task GetContactsAsync()
{
// Your Dynamics Web API URL
string api = “https://docmigrate.api.crm.dynamics.com/api/data/v9.0/&#8221;;

AuthenticationParameters ap = AuthenticationParameters.CreateFromResourceUrlAsync(new Uri(api)).Result;

// Set ‘Application ID’ and ‘Key’ generated from Azure
var creds = new ClientCredential(“e4ac3a78-xxxx-403a-a94c-xxxxxxx”, “hEo/xxxxxxxS+LEiYHpxxxxxxxRe8xg0=”);

AuthenticationContext authContext = new AuthenticationContext(ap.Authority);
var token = authContext.AcquireTokenAsync(ap.Resource, creds).Result.AccessToken;

using (HttpClient httpClient = new HttpClient())
{
httpClient.Timeout = new TimeSpan(0, 2, 0);
httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue(“Bearer”, token);

// Retrieve Top 1 Contact
HttpResponseMessage response = await httpClient.GetAsync(api + “/contacts?$top=1”);

// Parse the response
if (response.IsSuccessStatusCode)
{
JObject contact = JsonConvert.DeserializeObject<JObject>(await response.Content.ReadAsStringAsync());

var contactName = contact.GetValue(“fullname”);

}
}
}

App User - 13

🙂

Dynamics 365 Spring Release 2018

Microsoft has released Spring ’18 release notes for Microsoft Business Applications and can be downloaded from here

D365_Spring_18

Refer this link for all latest updates on Dynamics 365

Dynamics Portals – Hide Search Control and clear cache

Other day, we got a requirement to hide the ‘Search’ control from our Custom portal.

Search-1

Its very easy to achieve by changing a configuration.

Steps to hide Search control:

  • Go to Portals ->Site Settings ->Search/Enabled record
  • By default the ‘Value’ will be ‘true’, to hide the ‘Search’ change to ‘false’

Search-5

  • Save
  • Refresh the browser and you should not see the ‘Search’ control

Search-2

What if you are still seeing the ‘Search’ control:

Sometimes for some reason (Mostly due to caching), you wont see the change immediately.

In this case you can either reset portal/clear the server side cache by following steps below.

Reset the Portal:

  • If the change is not taken place for all users you might want to reset the portal.
  • Resetting the portal is not feasible option especially when your portal is being tested, as the end users will experience a snag for sometime.
  • But if the Portal is in Development phase and you can reset from your ‘D365 Admin Center’
    • Go to ‘Admin Center’ -> Applications
    • Select your Portal application and click ‘Manage’

Search-4

  • Select ‘Portal Actions’ -> Reset

Search-3

Clear Server Side Cache:

  • You can force the portal to refresh its cache immediately.
  • To clear the server-side cache, sign in to the portal with Administrator web role
  • Navigate to the URL as follows : <Your_portal_URL>/_services/about
    • (i.e., You need to append /_services/about end of your portal URL)
  • Select Clear Cache.
  • Refer this for article more details

🙂