Thursday, January 28, 2016

How to implement 'LIKE' operator in Talend

In here ‘LIKE’ operator means similar to SQL like command.
In tMap filter expression you can use contains function from Java.
For example, if you need to filter only the row contains “uma”. Similar way you can use Java pattern matching code also.
row2.ScheduleDay.contains (“uma”) && row2.SheduleTime.contains (“uma”)

You also can use context variable for the expression
row2.ScheduleDay.contains(context.Day)) &&  row2.SheduleTime.contains(context.Hour) ))

Cheers! Uma

Thursday, January 14, 2016

How to create a Stored Procedure with Try Catch and Transaction

CREATE PROCEDURE dbo.spname
AS
BEGIN
BEGIN TRANSACTION;

BEGIN TRY

/*
YOUR COMMAND
/*

END TRY

BEGIN CATCH

IF @@TRANCOUNT > 0
       ROLLBACK TRANSACTION;
END CATCH;

IF @@TRANCOUNT > 0
   COMMIT TRANSACTION;


END;

Cheers!
Uma

Sunday, January 10, 2016

How to connect Oracle 8 using Oracle SQLDeveloper

Latest version Oracle Developer is not support to connect Oracle 8. You could try using a much older version of SQL Developer to access 8, however most of the cases this might not work.
You can connect Oracle 8 via DBVisulaizer with Talend jar file.
DbVisualizer is a database management and analysis tool for all major databases (e.g. Oracle, SQL Server, DB2, Sybase, MySQL, SQLite). There is both a free version and a paid version.
The below image shows the location of jar file in Talend for Oracle 8
The below steps needs to follow to connect Oracle 8 using DBVisualizer
Cheers! Uma

Saturday, January 9, 2016

Split multiple values in one field into rows in SSIS

In some cases, one filed might have multiple values with comma separated, you might need to normalized and make it as multiple records. In this situation, SSIS script component gives more flexibility to split into multiple records or normalize the records.
  public override void Input0_ProcessInputRow(Input0Buffer Row)
   {
       string s = Row.sscale;
       var primeArray = s.Split(',');
       int indice = 1;


       for (int i = 0; i < primeArray.Length; i += 1)
       {
           string value = primeArray[i];
           //value = value.Replace(",",".");
           Output0Buffer.AddRow();
           Output0Buffer.scalevalue = primeArray[i];
           Output0Buffer.idcartola = Row.idcartola;
           Output0Buffer.courseid = Row.course;
           Output0Buffer.scaleid = Row.idscale;
           Output0Buffer.scaleindex = indice;
           Output0Buffer.scalevalue = value;
           Output0Buffer.actid = Row.idact;
           Output0Buffer.moodleinst = instance;
               
               indice = indice + 1;

       }
Cheers!
Uma

When or how to use tPrejob and tPostjob in Talend

The tPrejob and tPostjob components are designed to make the execution of tasks before and after a given job easier to manage.

These components differ from other components in that they do not actually process data and they do not have any components properties to be configured. A key feature of these components is that they are always guaranteed to be executed, even if the main data Job fails. Therefore, they are very useful for setup and teardown actions for a given Job.
Tasks that require the use of a tPrejob component include:
  • Loading context information required for the subjob execution.
  • Opening a database connection.
  • Making sure that a file exists.
  • Tasks that require the use of a tPostjob component include:
  • Cleaning up temporary files created during the processing of the main data Job.
  • Closing a database connection or a connection to an external service.
  • Any task required to be executed, even if the preceding Job or subjobs failed.

The following screenshots show few example of usage of tPrejob and tPostjob

https://help.talend.com/images/54/bk-tp-ds-studio-ug-542/PrepostJob1.png
http://www.talendbyexample.com/images/component-reference/talend-tprejob-tpostjob-components-01.png
Cheers!
Uma

How to validate schema in Talend using tSchemaCompileceCheck

Validates all input rows against a reference schema or check types, nullability, length of rows against reference values. The validation can be carried out in full or partly
The tSchemaComplianceCheck is a very useful component for ensuring that the data passing downstream is correct with respect to the defined schema.
The tFileInputDelimited component will detect only some of the anomalies within the data, whereas the tSchemaComplianceCheck component will perform a much more thorough validation of the data. If you look at the output, you will see the log entry, which shows that the name field has exceeded the maximum for the schema.
This simple exercise demonstrates how rows can be rejected using this component.
In the reject output, you can noticed that ErrorMessage “Exceed max legnth”.
Cheers! Uma

How to use bcp Utility MSSQL Server using Windows Authentication and SQL Server Authentication

The bcp utility bulk copies data between an instance of Microsoft SQL Server and a data file in a user-specified format. The bcp utility can be used to import large numbers of new rows into SQL Server tables or to export data out of tables into data files.
This examples shows how to use BCP command to transfer data from table to File and then from file to table.
Here I have created empty BCP_ Customers table similar structure of Customers stable.


Now let’s see, how to use BCP utility to Export Data from Table to file using Trusted Connection or Windows Authorization.


>BCP TESTDB.dbo.Customers OUT C:\bcp\Customers.txt -T –c


Arguments


in data_file | out data_file | queryout data_file | format nul
Specifies the direction of the bulk copy, as follows:
  • in copies from a file into the database table or view.
  • out copies from the database table or view to a file. If you specify an existing file, the file is overwritten. When extracting data, note that the bcp utility represents an empty string as a null and a null string as an empty string.
  • queryout copies from a query and must be specified only when bulk copying data from a query.
  • format creates a format file based on the option specified (-n, -c, -w, or -N) and the table or view delimiters. When bulk copying data, the bcp command can refer to a format file, which saves you from re-entering format information interactively. The formatoption requires the -f option; creating an XML format file, also requires the -x option. For more information, see Create a Format File (SQL Server). You must specify nul as the value (format nul).
-c
Performs the operation using a character data type. This option does not prompt for each field; it uses char as the storage type, without prefixes and with \t (tab character) as the field separator and \r\n (newline character) as the row terminator. -c is not compatible with -w.
-T
Specifies that the bcp utility connects to SQL Server with a trusted connection using integrated security. The security credentials of the network user, login_id, and password are not required. If –T is not specified, you need to specify –U and –P to successfully log in.

You can find the detail arguments detils in the follwing link:
https://msdn.microsoft.com/en-us/library/aa337544.aspx

You can see the output file in the output folder

Now let’s see, how to use BCP utility to Export Data from Table to file using SQLServer Authentication or security credentials.


BCP utility will helps if it finds any appropriate command


>BCP TESTDB.dbo.Customers OUT C:\bcp\Customers1.txt -Slocalhost -Usa -P123 –c


-S: Server Name (localhost)
-U: Username (sa)
-P: Password (123)
Now let’s see, how to use BCP utility to Import Data from File to Table using Server Authentication or security credentials.
>BCP TESTDB.dbo.BCP_Customers IN C:\bcp\Customers.txt -T -c

Now let’s see, how to use BCP utility to Export Data from File to Table using SQLServer Authentication or security credentials.
>BCP TESTDB.dbo.BCP_Customers IN C:\bcp\Customers1.txt -Slocalhost -Usa -P123 –c


Now let’s see, how to use BCP utility using SQL Query Export Data from Table to File using SQLServer Authentication or security credentials.
>BCP "SELECT firastname,lastname FROM TESTDB.dbo.BCP_Customers where id<4" queryout C:\bcp\Customersqueryout.txt -T –c


Cheers!
Uma

Do I need use a comma or semicolon to send the same email to multiple recipients Talend Job for tSendEmail component?

You should use to send emails to multiple address separated by semi-colons.

Cheers!
Uma

Monday, January 4, 2016

How to use XPath in XML Task in SSIS 2014

The XML Task is used when you need to validate, modify, extract, or even create files in an XML format.
In SSIS 2014, there are six operation types are available.


XPath: This option uses the XML Path Language and allows the extraction of sections or specific nodes from the structure of the XML document. You might use this option to extract data from the XML document prior to using the content. For example, you might want to pull out only the orders for a specific customer from an XML file.
In XPath, you can choose one of the operation in the below list box. Node list will retrieve all the nodes based on the XPath and Values will retrieve only values based on the XPath.
In this following example shows how to retire the nodes
The source file contains 3 nodes for book
Configure the XPath as //book and other file path variables and run. To learn about XPath, please follow this link

In the output you can notice that only book elements are retrieved.
Now let see the Value retrieve
If you need to retrieve all the authors first name
If you need to retrieve attribute you need to use @ symbol before the attribute, for example if you need to take all the style of the book
Output:
If you need to take last node:


Cheers!
Uma

How to use Diff and Patch in XML Task in SSIS 2014

Diff: This option uses difference algorithms to compare two XML documents to produce a third document called an XML Diffgram that contains the differences between them. Use this option with another XML Task using the Patch option to produce a smaller subset of data to insert into your data store. An example use of this task is extracting only the prices that have changed from a new price sheet in XML format.

Patch: This option applies the results of a Diff operation to an XML document to create a new XML document.


The following example show that how to use Diff operation type.
Create a two XML file with some data difference something like shown below.
Set the configuration as shown below and run the job
You will get two output files, one for the operation result is TRUE or FALSE and other one data difference.
Diff.xml
DiffData.xml
Patch


This option applies the results of a Diff operation to an XML document to create a new XML document.
In this example let’s see how to patch the above Diff example output.
Set the configuration and run.
You will notice that the output file will contains the difference as well.
Cheers!
Uma