Bite Size C# – XLS to CSV Converter

Let us have a look at my XLS to CSV Converter class today.

It is a static class, which is located in the Core.CSV namespace. It contains one static method CovertExcelToCsv which takes 3 parameters excelFilePath, csvOutputFile and worksheetNumber. The excelFilePath parameter is used to input the source Excel file, csvOutputFile is used to input the destination CSV file and lastly worksheetNumber is used to define which worksheet to convert, as CSV files do not have worksheets only one worksheet can be converted to CSV file at a time. The worksheetNumber parameter has a default value of 1 in the event that no value is passed in.

using System;
using System.Data;
using System.Data.OleDb;
using System.IO;

namespace Core.CSV
{
 public static class XlsToCsvConverter
 {
  public static void CovertExcelToCsv(string excelFilePath, string csvOutputFile, int worksheetNumber = 1)
  {
   if (!File.Exists(excelFilePath)) throw new FileNotFoundException(excelFilePath);
   if (File.Exists(csvOutputFile)) throw new ArgumentException("File exists: " + csvOutputFile);

   // Connection string
   var connectionStr =
    String.Format(
     "Provider=Microsoft.Jet.OLEDB.4.0;Data Source={0};Extended Properties=\"Excel 8.0;IMEX=1;HDR=NO\"",
     excelFilePath);
   var cnn = new OleDbConnection(connectionStr);

   // Get schema information and then data in spreadsheet
   var dt = new DataTable();
   try
   {
    cnn.Open();
    var schemaTable = cnn.GetOleDbSchemaTable(OleDbSchemaGuid.Tables, null);
    if (schemaTable != null && schemaTable.Rows.Count < worksheetNumber)
     throw new ArgumentException("The worksheet number requested does not exist.");
    if (schemaTable != null)
    {
     var worksheet = schemaTable.Rows[worksheetNumber - 1]["table_name"].ToString().Replace("'", "");
     var sql = String.Format("select * from [{0}]", worksheet);
     var da = new OleDbDataAdapter(sql, cnn);
     da.Fill(dt);
    }
   }
   catch (Exception ex)
   {
    //Insert logging of exception here
    throw;
   }
   finally
   {
    // Close connection to the spreadsheet
    cnn.Close();
   }

   // Write out CSV data
   using (var wtr = new StreamWriter(csvOutputFile))
   {
    foreach (DataRow row in dt.Rows)
    {
     var firstLine = true;
     foreach (DataColumn col in dt.Columns)
     {
      if (!firstLine)
      {
       wtr.Write(",");
      }
      else
      {
       firstLine = false;
      }
      var data = row[col.ColumnName].ToString().Replace("\"", "");
      wtr.Write(String.Format("{0}", data));
     }
     wtr.WriteLine();
    }
   }
  }
 }
}
Bite Size C# – XLS to CSV Converter

Bite Size SQL – Basic Query Optimisation

How SQL queries are written can have a huge impact on how quickly and efficiently they execute. In this post I will explain some very basic methods that can be used to optimise SQL queries.

  • When writing select queries, only select the columns you need. In general using “SELECT *” is very bad practice.
  • Avoid using sub queries, rather use joins. What is meant by this is instead of using:
        SELECT COLUMNA FROM TABLEA WHERE COLUMNB IS IN (SELECT COLUMNB FROM TABLEB)
        Rather use:
        SELECT COLUMNA FROM TABLEA AS TA INNER JOIN TABLEB AS TB ON TA.COLUMNB = TB.COLUMNB
  • Use WHERE to limit the result set to only what you need. There is no point in pulling 10000 records if you are only looking for records, for example in a certain date range.
  • Use WITH NOLOCK with SELECT queries when feasible, this can prevent deadlocks which can cause serious performance issues. However note that using WITH NOLOCK can result in a non-accurate result set being returned which contains non committed records, so keep this in mind when using. But when querying a production system database this risk might be outweighed by the risk of deadlocks occuring.

One last point on stored procedures, this is not really an optimisation point as much as a best practice point. When developing stored procedures always write them to return result sets of the same dimensions. I have worked on numerous legacy systems were this was not the case and it results in a complete nightmare, especially with projects that utilise an ORM.

Bite Size SQL – Basic Query Optimisation

Bite Size C# – CSV File Creator

Today we will be looking at my CSV Creator class, which is included in my Core.dll library.

It has 1 static method that takes a generic list of objects and a CSV file name. It uses the generic list of objects to construct the CSV file. The properties of the object type contained in the list are used in the creation of the header row of the file. The CSV file name parameter is used to save the created file to disk.

This class in contained in my Core.CSV namespace. Here is the code:

using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Reflection;

namespace Core.CSV
{
    public static class CsvCreator
    {
        public static void CreateCsvFromGenericList<T>(List<T> list, string csvNameWithExt)
        {
            if (list == null || list.Count == 0) return;

            //get type of objects in list
            Type t = list[0].GetType();
            string newLine = Environment.NewLine;

            using (var sw = new StreamWriter(csvNameWithExt))
            {
                //Create an instance of the class
                object o = Activator.CreateInstance(t);
                //Get all properties from class created, this will be used for CSV header row
                PropertyInfo[] props = o.GetType().GetProperties();

                //Create header row based on properties of class
                foreach (
                    PropertyInfo prop in
                        props.Where(prop => prop.Name.ToUpper() != "ENTITYSTATE" 
                        && prop.Name.ToUpper() != "ENTITYKEY"))
                {
                    sw.Write(prop.Name.ToUpper() + ",");
                }
                sw.Write(newLine);

                //create data rows using each object in list
                foreach (T item in list)
                {
                    foreach (string rowData in from pi in props
                                               where !Convert.ToString(item.GetType()
                                                        .GetProperty(pi.Name)
                                                        .GetValue(item, null)).Contains("Unchanged") &&
                                                     !Convert.ToString(item.GetType().GetProperty(pi.Name)
                                                                           .GetValue(item, null))
                                                             .Contains("System.Data.EntityKey")
                                               select Convert.ToString(item.GetType()
                                                                           .GetProperty(pi.Name)
                                                                           .GetValue(item, null))
                                                             .Replace(',', ' ') + ',')
                    {
                        sw.Write(rowData);
                    }
                    sw.Write(newLine);
                }
            }
        }
    }
}

This class is useful when you simply want to create a CSV file from a generic object list quickly.

Bite Size C# – CSV File Creator

Bite Size C#

As a software developer I believe it is critical to build reusable code when possible. All software engineers have their toolbox of previously written code that they reuse when the need arises. Keeping this in mind I have built a Class Library, that I continually extend, that contains all the reusable functions that I use across multiple projects. I call this my Core.dll and include it in all the projects I work on.

The project contains multiple classes for a variety of functions, anything from string manipulation to file generation. I will be sharing little code bits from this Core library in the Bite Size C# posts.

Today we will look at my StringExtensions class, which falls in my Core.Extensions namespace.

using System;

namespace Core.Extension
{
    public static class StringExtensions
    {
        public static bool Contains(this string source, string toCheck, StringComparison comp)
        {      
            return source.IndexOf(toCheck, comp) >= 0;
        }

        public static string GetLast(this string source, int tailLength)
        {
            return tailLength >= source.Length ? source : source.Substring(source.Length - tailLength);
        }
    }
}

The Class contains 2 methods:

A Contains function that is similar to the built-in string.Contains method except that it takes a StringComparison parameter so that case sensitivity, culture and sort rules can be configured.

A GetLast method that passes back the requested last bit of a string (for example last 4 characters of a string.)

When this class is included in a project the following operations can be performed:

 private static void Main(string[] args)
        {
            string testString = "This is a test string";
            bool test = testString.Contains("STRING", StringComparison.InvariantCultureIgnoreCase);
                //This will return true as ignore case is set


            string resultString = -testString.GetLast(6);
                //This will return "string", the last 6 characters in the string
        }

This class acts as an extension class to the predefined C# string class, so its methods appear in the class method list exactly like the built-in string method do (such as string.Equals or string.Trim).

Bite Size C#

Bite size SQL

I have a few SQL scripts that I tend to reuse quite often. I will post one or two of these scripts from time to time. Just note that <TABLE>, <COLUMN>, <CONDITION>, <UNIQUE IDENTIFIER COLUMN>, etc. are place holders and must be replaced with a table name, column name and so forth as required.

Insert auto incrementing numbers into a table:

DECLARE @VAL INT;
SET @VAL = 0;
UPDATE <TABLE>
SET @VAL = <COLUMN> = @VAL + 1
WHERE <CONDITION>;

Remove duplicate rows from a table:

DELETE FROM <TABLE>
WHERE <UNIQUE IDENTIFIER COLUMN> NOT IN
(
SELECT MAX (<UNIQUE IDENTIFIER COLUMN>)
FROM <TABLE>
GROUP BY <DUPLICATE COLUMN 1>,<DUPLICATE COLUMN 2>,<DUPLICATE COLUMN n>
)

Bite size SQL