Showing posts with label programatic. Show all posts
Showing posts with label programatic. Show all posts

Monday, February 20, 2012

programatic transferdatabasetask in ssis

I am trying to code a package that runs a transferdatabasetask with the following code

Dim package As New Package()

package.PackageType = DTSPackageType.DTSDesigner90

package.Name = "transfer db task"

package.Description = "transfer db task"

package.CreatorComputerName = System.Environment.MachineName

package.CreatorName = System.Environment.UserName

Dim dest As ConnectionManager = package.Connections.Add("OLEDB")

dest.Name = "Dest"

dest.ConnectionString = "Data Source=NSW97V9F1S\NSW97V9F1S;Initial Catalog=RGTemp;User Id=rgTest;Password=12345"

'dest.ConnectionString = "SqlServerName=PDNCNLNJ1S\SQLSERVER2005;UseWindowsAuthentication=True;UserName=sa;"

Dim source As ConnectionManager = package.Connections.Add("OLEDB")

source.Name = "Source"

source.ConnectionString = "Data Source=NSW97V9F1S\NSW97V9F1S;Initial Catalog=RGTemp;User Id=rgTest;Password=12345"

'source.ConnectionString = "SqlServerName=NSW97V9F1S\NSW97V9F1S;UseWindowsAuthentication=True;UserName=;"

Dim th As TaskHost = TryCast(package.Executables.Add("STOCK:TransferDatabaseTask"), TaskHost)

th.Name = "transfer db task"

th.Description = "The transfer task"

th.Properties("Action").SetValue(th, 0) '0: copy

th.Properties("Method").SetValue(th, 1) '1: destination online

th.Properties("DestinationConnection").SetValue(th, dest.ID)

th.Properties("DestinationDatabaseFiles").SetValue(th, "rgTemp1.mdf,C:\Program Files\Microsoft SQL Server\MSSQL.1\MSSQL\DATA,'';rgTemp1_log.ldf,C:\Program Files\Microsoft SQL Server\MSSQL.1\MSSQL\DATA,''")

'th.Properties("DestinationDatabaseFiles").SetValue(th, "C:\Program Files\Microsoft SQL Server\MSSQL.1\MSSQL\DATA\rgTemp1.mdf;C:\Program Files\Microsoft SQL Server\MSSQL.1\MSSQL\DATA\rgTemp1_log.ldf")

th.Properties("DestinationDatabaseName").SetValue(th, "RGTemp1")

th.Properties("DestinationOverwrite").SetValue(th, True)

th.Properties("ReattachSourceDatabase").SetValue(th, False)

th.Properties("SourceConnection").SetValue(th, source.ID)

th.Properties("SourceDatabaseFiles").SetValue(th, "'rgTemp.mdf','C:\Program Files\Microsoft SQL Server\MSSQL.1\MSSQL\DATA','';'rgTemp_log.ldf','C:\Program Files\Microsoft SQL Server\MSSQL.1\MSSQL\DATA',''")

MsgBox(th.Properties("SourceDatabaseFiles").GetValue(th).ToString)

MsgBox(th.Properties("DestinationDatabaseFiles").GetValue(th).ToString)

th.Properties("SourceDatabaseName").SetValue(th, "RGTemp")

Dim status As DTSExecResult = package.Validate(Nothing, Nothing, Nothing, Nothing)

' If the package validated successfully, then execute it.

If status = DTSExecResult.Success Then

' Execute the package

Dim result As DTSExecResult = package.Execute(Nothing, Nothing, Nothing, Nothing, Nothing)

End If

'Dts.TaskResult = Dts.Results.Success

End Sub

The problem is that the package validation fails and the 'execute' statement never runs. the two message boxes report zero length strings in the source and destination files properties. It seems that the hard coded filenames that I have provided are not correct.

Can anyone shed any light on what is wrong here:

regards

Ray

What is the error message on validation?

When you add connection manager, the type should be "SMOServer" instead of "OLEDB". So, use something like:
package.Connections.Add("SMOServer")

Setting the properties is easier if you get the InnerObject from task host as in:
TransferDatabaseTask task = (TransferDatabaseTask) th.InnerObject;

For source and destination connections, you should set the name of the connection manager instead of ID. (I am not sure if using the ID is correct)

You have to escape '\' and " in the values for source and destination database file as
"\"rgTemp.mdf\",\"C:\\Program

Files\\Microsoft SQL

Server\\MSSQL.1\\MSSQL\\DATA\",\"\";"\"rgTemp.ldf\",\"C:\\Program

Files\\Microsoft SQL

Server\\MSSQL.1\\MSSQL\\DATA\",\"\";|||

Thanks for the reply

I tried your suggestions and got as follows

Using SMOServer invalidated the connection strings and I could only set them in a format that doesn't allow SQL Server login or a password

I escaped the file names in VB as double double quotes and that fixed the source and destination file name assignments

But the routine still returns 'failed' from the 'validate' call. there is no error message the 'validate' routine just returns 'failed' with no exception or message that I can find.

if i comment out the validate call the 'execute' call just returns failed with no exception.

I am not getting any help from the system here.

we have made a step forward here thanks to you but I am still stuck

Do you have any other ideas?

regards

Ray

|||To create connection manager you can use the following code:
connectionString = String.Format("SqlServerName={0};UseWindowsAuthentication=true;", serverName);
//connectionString = String.Format("SqlServerName={0};UseWindowsAuthentication=false;UserName={1};Password={2}",serverName, userName, passwd);

ConnectionManager connectionManager = package.Connections.Add("SMOServer");
connectionManager.ConnectionString = connectionString;
connectionManager.Name = connectionManagerName;

Programatic replication

I have SELECT access to a database that is very, very slow to work with to
the point where I am looking to replicate at least one table to my own
server to use the data locally. What is the best way to check for new
records if the table key is not incremental? I prefer not to try a "If
exists" on the entire databse every minute or due to the speed issues.Hi
If you are allowed to change the structure of the table then you may want to
add a rowversion column. If not maybe you should use log shipping to recreat
e
the whole databases?
John
"Dave S." wrote:

> I have SELECT access to a database that is very, very slow to work with to
> the point where I am looking to replicate at least one table to my own
> server to use the data locally. What is the best way to check for new
> records if the table key is not incremental? I prefer not to try a "If
> exists" on the entire databse every minute or due to the speed issues.
>
>

Programatic or expression for failure

Hello,

Is there a way is a script task or via a procedural expression to tell if a package is going to fail because of an error?

This would be used to fire something which should occure if the max number of errors/error severity to fail the package had been reached

Thanks

Paul

Hi Paul,

I'm unaware of a way to tell before execution that a package is going to encounter n number of errors. You can use the DtExec /Validate switch (or check the Validate package without executing checkbox in DtExecUI) to perform a package validation, but validation will not catch all conditions that can raise errors during execution.

You may want to check into SSIS Event Handlers. These allow you to respond to error conditions.

Hope this helps,

Andy

|||

Sorry I did not clarify the problem very well.

When a package is running and an OnError event handeler is called i want it to do somthing specific if the package will fail because of the error. usually a OnError would cause the package to fall over but i have changed the package to allowed errors to be 3 so not every error would cause the package to fail.

e.g.

Error 1 would send an email

Error 2 would send an email

Error 3 would send an email and move a file (the package would fail at this point)

|||

Hi Paul,

Oh, ok. Count the errors.

Create a package-scoped variable called iErrorCount of Int32 data type. Make the first task in your OnError Event Handler a Script Task that contains code similar to the following:

Code Snippet

Dim iErrCount as Integer =

Convert.ToInt32(

Dts.Variables("iErrorCount").Value)

iErrCount += 1

If iErrCount >= 3 Then

...

Hope this helps,

Andy

Programatic Import (Space delimited) import into SQL Server 2000

I am trying to find a way to do a programatic import of a space delimited
file into a SQL Server 2000 database.
The file is actually an IIS web log file. I can run through the file and
strip out the header information, I can even create a seperate file for each
change in header information is needed.
The problem becomes importing that file into the database. The log files are
on average 6MB each. The import will need to run daily on just a single log
file (after previous log files have been imported)
I do not want to use BULK INSERT because I have had a lot of inconsistant
problems with that functionality in the past.
Does anyone have any ideas?
John WycoffI would start with the DTS data import wizard. You can specify a text
file as input, and space as the column delimiter, and the package it
generates can be saved and modified later. It does assume that all
the rows are in the same format.
Roy Harvey
Beacon Falls, CT
On Thu, 20 Jul 2006 07:15:01 -0700, John Wycoff <John
Wycoff@.discussions.microsoft.com> wrote:
>I am trying to find a way to do a programatic import of a space delimited
>file into a SQL Server 2000 database.
>The file is actually an IIS web log file. I can run through the file and
>strip out the header information, I can even create a seperate file for each
>change in header information is needed.
>The problem becomes importing that file into the database. The log files are
>on average 6MB each. The import will need to run daily on just a single log
>file (after previous log files have been imported)
>I do not want to use BULK INSERT because I have had a lot of inconsistant
>problems with that functionality in the past.
>Does anyone have any ideas?
>John Wycoff|||Use DTS. If the file is 'space' delimited, or is fixed position, DTS can
handle the import. You can create a DTS 'Package' (the set of instructions),
and then schedule that Package to run when appropriate.
--
Arnie Rowland
Most good judgment comes from experience.
Most experience comes from bad judgment.
- Anonymous
"John Wycoff" <John Wycoff@.discussions.microsoft.com> wrote in message
news:170BB6CA-D3E7-4BEA-A336-390BF6A3DF9E@.microsoft.com...
> I am trying to find a way to do a programatic import of a space delimited
> file into a SQL Server 2000 database.
> The file is actually an IIS web log file. I can run through the file and
> strip out the header information, I can even create a seperate file for
> each
> change in header information is needed.
> The problem becomes importing that file into the database. The log files
> are
> on average 6MB each. The import will need to run daily on just a single
> log
> file (after previous log files have been imported)
> I do not want to use BULK INSERT because I have had a lot of inconsistant
> problems with that functionality in the past.
> Does anyone have any ideas?
> John Wycoff|||The problem is that I do not know what the filenames will be. And the formats
will be different. The closest I can get is to run through the log files
generating a seperate log file for every different format. But I cannot
programattically change the DTS package everytime, nor can I tell the DTS
package to import every file in a specific directory.|||On Tue, 25 Jul 2006 06:08:02 -0700, John Wycoff
<JohnWycoff@.discussions.microsoft.com> wrote:
>The problem is that I do not know what the filenames will be. And the formats
>will be different. The closest I can get is to run through the log files
>generating a seperate log file for every different format. But I cannot
>programattically change the DTS package everytime, nor can I tell the DTS
>package to import every file in a specific directory.
I am still pretty unclear about what the files look like. How many
different tables is the data being loaded into? Do the headers have
any data you need? Is it one header per file? Or a header, followed
by several items, then another header and more items?
It IS possible to get DTS to import every file in a folder. There is
VBA coding involved.
Roy Harvey
Beacon Falls, CT

Programatic Import (Space delimited) import into SQL Server 2000

I am trying to find a way to do a programatic import of a space delimited
file into a SQL Server 2000 database.
The file is actually an IIS web log file. I can run through the file and
strip out the header information, I can even create a seperate file for each
change in header information is needed.
The problem becomes importing that file into the database. The log files are
on average 6MB each. The import will need to run daily on just a single log
file (after previous log files have been imported)
I do not want to use BULK INSERT because I have had a lot of inconsistant
problems with that functionality in the past.
Does anyone have any ideas?
John WycoffI would start with the DTS data import wizard. You can specify a text
file as input, and space as the column delimiter, and the package it
generates can be saved and modified later. It does assume that all
the rows are in the same format.
Roy Harvey
Beacon Falls, CT
On Thu, 20 Jul 2006 07:15:01 -0700, John Wycoff <John
Wycoff@.discussions.microsoft.com> wrote:

>I am trying to find a way to do a programatic import of a space delimited
>file into a SQL Server 2000 database.
>The file is actually an IIS web log file. I can run through the file and
>strip out the header information, I can even create a seperate file for eac
h
>change in header information is needed.
>The problem becomes importing that file into the database. The log files ar
e
>on average 6MB each. The import will need to run daily on just a single log
>file (after previous log files have been imported)
>I do not want to use BULK INSERT because I have had a lot of inconsistant
>problems with that functionality in the past.
>Does anyone have any ideas?
>John Wycoff|||Use DTS. If the file is 'space' delimited, or is fixed position, DTS can
handle the import. You can create a DTS 'Package' (the set of instructions),
and then schedule that Package to run when appropriate.
Arnie Rowland
Most good judgment comes from experience.
Most experience comes from bad judgment.
- Anonymous
"John Wycoff" <John Wycoff@.discussions.microsoft.com> wrote in message
news:170BB6CA-D3E7-4BEA-A336-390BF6A3DF9E@.microsoft.com...
> I am trying to find a way to do a programatic import of a space delimited
> file into a SQL Server 2000 database.
> The file is actually an IIS web log file. I can run through the file and
> strip out the header information, I can even create a seperate file for
> each
> change in header information is needed.
> The problem becomes importing that file into the database. The log files
> are
> on average 6MB each. The import will need to run daily on just a single
> log
> file (after previous log files have been imported)
> I do not want to use BULK INSERT because I have had a lot of inconsistant
> problems with that functionality in the past.
> Does anyone have any ideas?
> John Wycoff

Programatic Fuzzy Lookup

I want to use Fuzzy lookup to search a table for similar matches. I would like to do this from code in Visual C#. How do I do this or can I? If I cannot, will this be something available in the future?You can run a package containing fuzzy lookups from C#. Do a search in help for "Running an existing package using Visual Basic". The code can easily be ported to C#.

What you can't do is take SSIS functionality out of SSIS and use it elsewhere - it must all run within a package.

-Jamie|||But can I pass in info such as a parameter into the package and have it run it based on that parameter?

For example, I want to look up any and all customers that match to 'Blue something' ('Blue Cross / Blue Shield', 'Blue Cros / Blue Shield', 'Blu Cross & Shield', etc.), so I want to pass in the parameter 'Blue' and it will execute and pull out any customer that fuzzies to anything relating to 'Blue'

I haven't been able to find in the documentation any way of doing this. I have seen a lot of 'hints' saying that you can do this, but nothing on how to do this.

Thanks|||

Tazzmann wrote:

But can I pass in info such as a parameter into the package and have it run it based on that parameter?

For example, I want to look up any and all customers that match to 'Blue something' ('Blue Cross / Blue Shield', 'Blue Cros / Blue Shield', 'Blu Cross & Shield', etc.), so I want to pass in the parameter 'Blue' and it will execute and pull out any customer that fuzzies to anything relating to 'Blue'

I haven't been able to find in the documentation any way of doing this. I have seen a lot of 'hints' saying that you can do this, but nothing on how to do this.

Thanks

Right, I get you now.

Hmm...yes and no really. Yes, you can pass values into a package quite easy but fuzzy lookup isn't designed to be used using values that exist in your package. It does a lookup of incoming data (e.g. from a table) against a lookup table. The last bit that I mentioned here is key, it looks up against a table NOT against a value that exists in your package which I think is what you want to do.

Why not put the value "Blue" into a table and lookup against that table. That would work fine.

Basically, yes, you can integrate fuzzy lookup functionality into other applications, but with the confines of the way the fuzzy components work in SSIS. you seem to want something slightly different, and it doesn't do that!

Does that make any sense at all?

-Jamie|||Yes Jamie that does make sense, thanks. I did find a way to kind of do what I want and it works good through the Business Intelligence designer, but I still have a couple of questions if you can be patient with a total newbie into this new SQL Server technology...

What I did is set up a package-wide variable called User::Test and placed the value 'Blue' into it. I then took the OLE DB Source component and changed the Data Access Mode to use SQL Command. I then Set the SQL Command Text to the following:

Select * From Tablename where ColumnName like ? + '%'

I also mapped a parameter called @.Input to User::Test variable

I then connected this to the Fuzzy Grouping (not Lookup, yet) and then put the output from that to OLE DB Destination source.

When ran, it gives me the expected results of all the matches for 'Blue'.

Now my questions are these:

1. When I execute this package from a Stored Procedure, can I pass a value to the parameter @.Input (variable User::Test) at runtime?

2. Do I need to set up another parameter / Variable to hold the results so that I may pass that back to the calling Stored Procedure or do I have to have the results dumped to a table?

I am making headway and I thank you for your help so far.

Tazz|||

Tazzmann wrote:

Yes Jamie that does make sense, thanks. I did find a way to kind of do what I want and it works good through the Business Intelligence designer, but I still have a couple of questions if you can be patient with a total newbie into this new SQL Server technology...

You're a beginner you say? Blimey, you progress quickly Smile

Tazzmann wrote:

What I did is set up a package-wide variable called User::Test and placed the value 'Blue' into it. I then took the OLE DB Source component and changed the Data Access Mode to use SQL Command. I then Set the SQL Command Text to the following:

Select * From Tablename where ColumnName like ? + '%'

I also mapped a parameter called @.Input to User::Test variable

I then connected this to the Fuzzy Grouping (not Lookup, yet) and then put the output from that to OLE DB Destination source.

When ran, it gives me the expected results of all the matches for 'Blue'.

Now my questions are these:

1. When I execute this package from a Stored Procedure, can I pass a value to the parameter @.Input (variable User::Test) at runtime?

Yes, I think you can (I should know how but I don't, sorry). Are you sure you want to execute this from a sproc tho - I thought you were calling it from .Net?

Tazzmann wrote:

2. Do I need to set up another parameter / Variable to hold the results so that I may pass that back to the calling Stored Procedure or do I have to have the results dumped to a table?

The ADO.Net DataReader destination enables your data to be consumed by .Net applications. Look up "Datareader Destination" in BOL for more info.

Tazzmann wrote:

I am making headway and I thank you for your help so far.

Tazz


This sounds like an interesting problem, good luck with it.

-Jamie|||Thanks for the compliment. Big Smile I have been programming for about 13 years and the last few years have been in .NET.

The 2005 version has seen a lot of improvements and changes especially in the SQL Server side of things. We just installed the Beta where I work so I am trying to get my head wrapped around it and am trying to do some proof of concept things such as what I am trying to accomplish above.

No, I don't have to call it from a stored proc, if I can call it from .NET I would like that much better (and so would our DBA's).

Thank you for the ADO.NET Datareader destination hint, that is exactly what I needed. You are a blessing in disguise. I will post back here if I either get it running or find that it is not possible to do what I am trying to do. So far it looks very possible.

Thanks again! Big Smile

Programatic cube partition creation?

How can you programmatically create cube partitions? I have a cube that
is 3.2GB in fact data size and a year should contain about 35GB of fact
data. I would like to partition it into several slices, however I do not
want to create 50 partition through the GUI. Help me please. Thanks.
Kevin Brooks
If this is AS2K5, then you can use XMLA scripting. Just create one partition
using SSMS, then right-click on it and generate an XMLA script. Then you can
modify the script for the other 49 partitions and run the script through
SSMS to create them.
If using AS2K, then there is a similar, but unsupport utility called DSO/XML
that is available for download here:
http://www.microsoft.com/downloads/d...DisplayLang=en
Dave Wickert [MSFT]
dwickert@.online.microsoft.com
Program Manager
BI Systems Team
SQL BI Product Unit (Analysis Services)
This posting is provided "AS IS" with no warranties, and confers no rights.
"K. Brooks" <jeepnreb@.yahoo.com> wrote in message
news:O0YxWP$CGHA.3820@.TK2MSFTNGP12.phx.gbl...
> How can you programmatically create cube partitions? I have a cube
> that is 3.2GB in fact data size and a year should contain about 35GB of
> fact data. I would like to partition it into several slices, however I do
> not want to create 50 partition through the GUI. Help me please. Thanks.
> Kevin Brooks
>

Programatic cube partition creation?

How can you programmatically create cube partitions? I have a cube that
is 3.2GB in fact data size and a year should contain about 35GB of fact
data. I would like to partition it into several slices, however I do not
want to create 50 partition through the GUI. Help me please. Thanks.
Kevin BrooksIf this is AS2K5, then you can use XMLA scripting. Just create one partition
using SSMS, then right-click on it and generate an XMLA script. Then you can
modify the script for the other 49 partitions and run the script through
SSMS to create them.
If using AS2K, then there is a similar, but unsupport utility called DSO/XML
that is available for download here:
http://www.microsoft.com/downloads/...&DisplayLang=en
--
Dave Wickert [MSFT]
dwickert@.online.microsoft.com
Program Manager
BI Systems Team
SQL BI Product Unit (Analysis Services)
--
This posting is provided "AS IS" with no warranties, and confers no rights.
"K. Brooks" <jeepnreb@.yahoo.com> wrote in message
news:O0YxWP$CGHA.3820@.TK2MSFTNGP12.phx.gbl...
> How can you programmatically create cube partitions? I have a cube
> that is 3.2GB in fact data size and a year should contain about 35GB of
> fact data. I would like to partition it into several slices, however I do
> not want to create 50 partition through the GUI. Help me please. Thanks.
> Kevin Brooks
>

Programatic creation of SSIS packages

I am playing around with creating packages using C#. One major problem I have run into is controlling the location of the tasks I create in the package. I have found no way to access any properties to set this. Does anyone know how to do this?

Dave,

I don't think the ability to define tis exists in the API at the moment. if you think it should then ask for it at Connect (http://connect.microsoft.com)

I recommend you take a read of this though:

Extended properties...
(http://sqljunkies.com/WebLog/knight_reign/archive/2005/01/13/6247.aspx)

-Jamie

|||Thank you, the extended properties look like they may do the trick.