My requirement is to load system generated CSV file into an SQL table. Below I'm providing the CSV file, table query and my code.
Below are the CSV files abc.csv and xyz.csv which column comes changing by column every time like in the second file as you can see last two column shuffled.
id,Name,name,Shift,Day,Night 1,ert,sdf,08/21/2017 Day,2,- 2,wer,asdf,08/21/2017 Day,-,1 3,rty,adsf,08/21/2017 Day,1,- 4,yui,adsf,08/21/2017 Day,-,- 5,qwe,asdf,08/21/2017 Day,-,- 6,ret,asdf,08/21/2017 Day,-,- 7,chh,asdf,08/21/2017 Day,-,- 8,sdf,cxvv,08/21/2017 Day,1,-
id,Name,name,Shift,Night,Day 1,ert,sdf,08/21/2017 Day,-,2 2,wer,asdf,08/21/2017 Day,1,- 3,rty,adsf,08/21/2017 Day,-,1 4,yui,adsf,08/21/2017 Day,-,- 5,qwe,asdf,08/21/2017 Day,-,- 6,ret,asdf,08/21/2017 Day,-,- 7,chh,asdf,08/21/2017 Day,-,- 8,sdf,cxvv,08/21/2017 Day,-,1
Below is the create table query to load this data in SQL:
CREATE DATABASE MES;
USE MES;
CREATE TABLE MESdata
(
Id int NOT NULL IDENTITY(1,1) PRIMARY KEY,
Employee_id int NOT NULL,
First_Name varchar(20) NOT NULL,
Last_Name varchar(20) NOT NULL,
Shift varchar(20) NOT NULL,
Day int NULL,
Night int NULL
);
Below is my attempt in PowerShell using Out-datatable.ps1 and write-datatable.ps1:
. C:\ReadCSV\Out-datatable.ps1;
. C:\ReadCSV\write-datatable.ps1;
if (Test-Path Variable:\my_import) {
Remove-Variable my_import
}
$csvfile = 'C:\ReadCSV\abc.csv'
$header = Get-Content 'C:\ReadCSV\abc.csv' | select -First 1
$sqlTable = 'MESdata'
$DataSource = 'HOST\INSTANCE'
$DataBase = 'MES'
$my_import = Import-Csv -UseCulture -Header $header $csvfile |
select @{Label="Employee_id";Expression={$_."id"}},
@{Label="First_Name";Expression={$_."Name"}},
@{Label="Last_Name";Expression={$_."name"}},
@{Label="Shift";Expression={$_."Shift"}},
@{Label="Day";Expression={$_."Day"}},
@{Label="Night";Expression={$_."Night"}} |
Out-DataTable
Write-DataTable -ServerInstance $DataSource -Database $DataBase -TableName $sqlTable -Data $my_import
#$ConnectionString ='Data Source={0}; Database={1}; Trusted_Connection=True;' -f $DataSource,$DataBase
#$bulkCopy = New-Object Data.SqlClient.SqlBulkCopy($ConnectionString)
#$bulkCopy.DestinationTableName = $sqlTable
#$bulkCopy.WriteToServer($my_import)
I got this first error:
Exception calling "WriteToServer" with "1" argument(s): "Column 'Employee_id'
does not allow DBNull.Value."
At C:\ReadCSV\stackoverflow.ps1:25 char:1
+ $bulkCopy.WriteToServer($my_import)
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [], MethodInvocationException
+ FullyQualifiedErrorId : InvalidOperationException
What should I do? Do I need to change the datatype of table fields but I don't want like that. But if I replace all not null to null then the script executed without error but with all NULL data.
-Header $headerand I suspect its causing your issue.