Wednesday 4 October 2017

Dynamic Schema Load in Talend

Scenario

I want to Load 10 table from database 1 to database 2. these 10 tables having different schema .
I need to read table schema dynamically from database 1 and create table and insert the data into database 2.

Here i am going to Read schema from each table Dynamically. Dynamic schema we can use for only for insert the data and we cant  perform any validation on data.

Please see the job design




  • Take mysqlinput connection component for source and target DB's and configure the settings like hostname,server,user,password and db name.
tMysqlTablelist
  • Take TmysqlTablelist component to retrieve all tables from the Source DB.To retrieve all table write query as 'table_name' like '%' in the component and configure the settings as below.



tMysqlinput
  • tMysqlInput reads a database and extracts fields based on a query.
Mention the query as below
" select  *  from  " +((String)globalMap.get("tMysqlTableList_2_CURRENT_TABLE"))+";" in the tmysqlinput component.

((String)globalMap.get("tMysqlTableList_2_CURRENT_TABLE")  variable which retrieves all tables in the Database.
  • click on the edit schema of  tMysqlInput component add a field like data and mention schema type as dynamic as mentioned  in the below screenshot.


tMysqloutput
  • Take tMysqloutput component and mention ((String)globalMap.get("tMysqlTableList_2_CURRENT_TABLE") in the table name slot. 
  • click on Sync columns.
Run the Job
job will read the table by table dynamically and load the data into target database.


***********************THANK YOU*****************************




java.io.IOException: org.eclipse.aether.deployment.DeploymentException: Failed to deploy artifacts: Could not transfer artifact Return code is: 400, ReasonPhrase: Repository does not allow updating assets: releases.

 HI, by Default in Nexus release branch will not allow redeploy of the Same version of Talend job. if we need to redeploy same job again int...