网站数据表怎么做,网站模板免费下载酒店管理系统,vs用户登录注册网站建设代码,域名查询网1381#xff0c;kettle下载 kettle本身是Java开发的开源项目#xff0c;由于某种原因#xff0c;目前官网已经不支持下载了#xff0c;可到我的CSDN中下载 下载完后#xff0c;还需要安装JDK环境以及对应的数据库驱动包#xff0c;需要注意的是 该kettle的版本较低#xff…1kettle下载 kettle本身是Java开发的开源项目由于某种原因目前官网已经不支持下载了可到我的CSDN中下载 下载完后还需要安装JDK环境以及对应的数据库驱动包需要注意的是 该kettle的版本较低如果你是用的 mysql数据库需要下载 5.4.7以下的驱动包下载地址参照以下
MySQL :: Download Connector/J
2一切准备就绪后就可以开始使用啦
假设现在有一个场景需要将本地的所有库同步到目标服务器上对应的数据库中
其中涉及到 数据表的自动创建 以及 数据的自动同步这里以全量同步为例。
实现步骤分为三大块定义一个入口 任务该任务中涵盖 start、库表初始化(删除表及数据)、全量同步、结束
3main 任务脚本内容
?xml version1.0 encodingUTF-8?
jobnamemain/namedescription /extended_description /job_version /directory//directorycreated_user-/created_usercreated_date2024/06/03 09:54:18.592/created_datemodified_user-/modified_usermodified_date2024/06/03 09:54:18.592/modified_dateparameters/parametersslaveservers/slaveserversjob-log-tableconnection /schema /table /size_limit_lines /interval /timeout_days /fieldidID_JOB/idenabledY/enablednameID_JOB/name/fieldfieldidCHANNEL_ID/idenabledY/enablednameCHANNEL_ID/name/fieldfieldidJOBNAME/idenabledY/enablednameJOBNAME/name/fieldfieldidSTATUS/idenabledY/enablednameSTATUS/name/fieldfieldidLINES_READ/idenabledY/enablednameLINES_READ/name/fieldfieldidLINES_WRITTEN/idenabledY/enablednameLINES_WRITTEN/name/fieldfieldidLINES_UPDATED/idenabledY/enablednameLINES_UPDATED/name/fieldfieldidLINES_INPUT/idenabledY/enablednameLINES_INPUT/name/fieldfieldidLINES_OUTPUT/idenabledY/enablednameLINES_OUTPUT/name/fieldfieldidLINES_REJECTED/idenabledY/enablednameLINES_REJECTED/name/fieldfieldidERRORS/idenabledY/enablednameERRORS/name/fieldfieldidSTARTDATE/idenabledY/enablednameSTARTDATE/name/fieldfieldidENDDATE/idenabledY/enablednameENDDATE/name/fieldfieldidLOGDATE/idenabledY/enablednameLOGDATE/name/fieldfieldidDEPDATE/idenabledY/enablednameDEPDATE/name/fieldfieldidREPLAYDATE/idenabledY/enablednameREPLAYDATE/name/fieldfieldidLOG_FIELD/idenabledY/enablednameLOG_FIELD/name/fieldfieldidEXECUTING_SERVER/idenabledN/enablednameEXECUTING_SERVER/name/fieldfieldidEXECUTING_USER/idenabledN/enablednameEXECUTING_USER/name/fieldfieldidSTART_JOB_ENTRY/idenabledN/enablednameSTART_JOB_ENTRY/name/fieldfieldidCLIENT/idenabledN/enablednameCLIENT/name/field/job-log-tablejobentry-log-tableconnection /schema /table /timeout_days /fieldidID_BATCH/idenabledY/enablednameID_BATCH/name/fieldfieldidCHANNEL_ID/idenabledY/enablednameCHANNEL_ID/name/fieldfieldidLOG_DATE/idenabledY/enablednameLOG_DATE/name/fieldfieldidJOBNAME/idenabledY/enablednameTRANSNAME/name/fieldfieldidJOBENTRYNAME/idenabledY/enablednameSTEPNAME/name/fieldfieldidLINES_READ/idenabledY/enablednameLINES_READ/name/fieldfieldidLINES_WRITTEN/idenabledY/enablednameLINES_WRITTEN/name/fieldfieldidLINES_UPDATED/idenabledY/enablednameLINES_UPDATED/name/fieldfieldidLINES_INPUT/idenabledY/enablednameLINES_INPUT/name/fieldfieldidLINES_OUTPUT/idenabledY/enablednameLINES_OUTPUT/name/fieldfieldidLINES_REJECTED/idenabledY/enablednameLINES_REJECTED/name/fieldfieldidERRORS/idenabledY/enablednameERRORS/name/fieldfieldidRESULT/idenabledY/enablednameRESULT/name/fieldfieldidNR_RESULT_ROWS/idenabledY/enablednameNR_RESULT_ROWS/name/fieldfieldidNR_RESULT_FILES/idenabledY/enablednameNR_RESULT_FILES/name/fieldfieldidLOG_FIELD/idenabledN/enablednameLOG_FIELD/name/fieldfieldidCOPY_NR/idenabledN/enablednameCOPY_NR/name/field/jobentry-log-tablechannel-log-tableconnection /schema /table /timeout_days /fieldidID_BATCH/idenabledY/enablednameID_BATCH/name/fieldfieldidCHANNEL_ID/idenabledY/enablednameCHANNEL_ID/name/fieldfieldidLOG_DATE/idenabledY/enablednameLOG_DATE/name/fieldfieldidLOGGING_OBJECT_TYPE/idenabledY/enablednameLOGGING_OBJECT_TYPE/name/fieldfieldidOBJECT_NAME/idenabledY/enablednameOBJECT_NAME/name/fieldfieldidOBJECT_COPY/idenabledY/enablednameOBJECT_COPY/name/fieldfieldidREPOSITORY_DIRECTORY/idenabledY/enablednameREPOSITORY_DIRECTORY/name/fieldfieldidFILENAME/idenabledY/enablednameFILENAME/name/fieldfieldidOBJECT_ID/idenabledY/enablednameOBJECT_ID/name/fieldfieldidOBJECT_REVISION/idenabledY/enablednameOBJECT_REVISION/name/fieldfieldidPARENT_CHANNEL_ID/idenabledY/enablednamePARENT_CHANNEL_ID/name/fieldfieldidROOT_CHANNEL_ID/idenabledY/enablednameROOT_CHANNEL_ID/name/field/channel-log-tablepass_batchidN/pass_batchidshared_objects_file /entriesentrynameSTART/namedescription /typeSPECIAL/typestartY/startdummyN/dummyrepeatN/repeatschedulerType0/schedulerTypeintervalSeconds0/intervalSecondsintervalMinutes60/intervalMinuteshour12/hourminutes0/minutesweekDay1/weekDayDayOfMonth1/DayOfMonthparallelN/paralleldrawY/drawnr0/nrxloc176/xlocyloc208/yloc/entryentryname同步aav/namedescription /typeTRANS/typespecification_methodfilename/specification_methodtrans_object_id /filename${Internal.Entry.Current.Directory}/sync_aav.ktr/filenametransname /arg_from_previousN/arg_from_previousparams_from_previousN/params_from_previousexec_per_rowN/exec_per_rowclear_rowsN/clear_rowsclear_filesN/clear_filesset_logfileN/set_logfilelogfile /logext /add_dateN/add_dateadd_timeN/add_timeloglevelBasic/loglevelclusterN/clusterslave_server_name /set_append_logfileN/set_append_logfilewait_until_finishedY/wait_until_finishedfollow_abort_remoteN/follow_abort_remotecreate_parent_folderN/create_parent_folderlogging_remote_workN/logging_remote_workrun_configurationPentaho local/run_configurationparameterspass_all_parametersY/pass_all_parameters/parametersparallelN/paralleldrawY/drawnr0/nrxloc640/xlocyloc48/yloc/entryentryname初始化所有表/namedescription /typeJOB/typespecification_methodfilename/specification_methodjob_object_id /filename${Internal.Entry.Current.Directory}/init.kjb/filenamejobname /arg_from_previousN/arg_from_previousparams_from_previousN/params_from_previousexec_per_rowN/exec_per_rowset_logfileN/set_logfilelogfile /logext /add_dateN/add_dateadd_timeN/add_timeloglevelNothing/loglevelslave_server_name /wait_until_finishedY/wait_until_finishedfollow_abort_remoteN/follow_abort_remoteexpand_remote_jobN/expand_remote_jobcreate_parent_folderN/create_parent_folderpass_exportN/pass_exportparameterspass_all_parametersY/pass_all_parameters/parametersset_append_logfileN/set_append_logfileparallelN/paralleldrawY/drawnr0/nrxloc352/xlocyloc208/yloc/entryentrynameDUMMY/namedescription /typeSPECIAL/typestartN/startdummyY/dummyrepeatN/repeatschedulerType0/schedulerTypeintervalSeconds0/intervalSecondsintervalMinutes60/intervalMinuteshour12/hourminutes0/minutesweekDay1/weekDayDayOfMonth1/DayOfMonthparallelN/paralleldrawY/drawnr0/nrxloc928/xlocyloc208/yloc/entryentryname同步aps/namedescription /typeTRANS/typespecification_methodfilename/specification_methodtrans_object_id /filename${Internal.Entry.Current.Directory}/sync_aps.ktr/filenametransname /arg_from_previousN/arg_from_previousparams_from_previousN/params_from_previousexec_per_rowN/exec_per_rowclear_rowsN/clear_rowsclear_filesN/clear_filesset_logfileN/set_logfilelogfile /logext /add_dateN/add_dateadd_timeN/add_timeloglevelBasic/loglevelclusterN/clusterslave_server_name /set_append_logfileN/set_append_logfilewait_until_finishedY/wait_until_finishedfollow_abort_remoteN/follow_abort_remotecreate_parent_folderN/create_parent_folderlogging_remote_workN/logging_remote_workrun_configurationPentaho local/run_configurationparameterspass_all_parametersY/pass_all_parameters/parametersparallelN/paralleldrawY/drawnr0/nrxloc640/xlocyloc128/yloc/entryentryname同步douban/namedescription /typeTRANS/typespecification_methodfilename/specification_methodtrans_object_id /filename${Internal.Entry.Current.Directory}/sync_douban.ktr/filenametransname /arg_from_previousN/arg_from_previousparams_from_previousN/params_from_previousexec_per_rowN/exec_per_rowclear_rowsN/clear_rowsclear_filesN/clear_filesset_logfileN/set_logfilelogfile /logext /add_dateN/add_dateadd_timeN/add_timeloglevelBasic/loglevelclusterN/clusterslave_server_name /set_append_logfileN/set_append_logfilewait_until_finishedY/wait_until_finishedfollow_abort_remoteN/follow_abort_remotecreate_parent_folderN/create_parent_folderlogging_remote_workN/logging_remote_workrun_configurationPentaho local/run_configurationparameterspass_all_parametersY/pass_all_parameters/parametersparallelN/paralleldrawY/drawnr0/nrxloc640/xlocyloc208/yloc/entryentryname同步sp-dev/namedescription /typeTRANS/typespecification_methodfilename/specification_methodtrans_object_id /filename${Internal.Entry.Current.Directory}/sync_sp-dev.ktr/filenametransname /arg_from_previousN/arg_from_previousparams_from_previousN/params_from_previousexec_per_rowN/exec_per_rowclear_rowsN/clear_rowsclear_filesN/clear_filesset_logfileN/set_logfilelogfile /logext /add_dateN/add_dateadd_timeN/add_timeloglevelBasic/loglevelclusterN/clusterslave_server_name /set_append_logfileN/set_append_logfilewait_until_finishedY/wait_until_finishedfollow_abort_remoteN/follow_abort_remotecreate_parent_folderN/create_parent_folderlogging_remote_workN/logging_remote_workrun_configurationPentaho local/run_configurationparameterspass_all_parametersY/pass_all_parameters/parametersparallelN/paralleldrawY/drawnr0/nrxloc640/xlocyloc368/yloc/entryentryname同步lottery/namedescription /typeTRANS/typespecification_methodfilename/specification_methodtrans_object_id /filename${Internal.Entry.Current.Directory}/sync_lottery.ktr/filenametransname /arg_from_previousN/arg_from_previousparams_from_previousN/params_from_previousexec_per_rowN/exec_per_rowclear_rowsN/clear_rowsclear_filesN/clear_filesset_logfileN/set_logfilelogfile /logext /add_dateN/add_dateadd_timeN/add_timeloglevelBasic/loglevelclusterN/clusterslave_server_name /set_append_logfileN/set_append_logfilewait_until_finishedY/wait_until_finishedfollow_abort_remoteN/follow_abort_remotecreate_parent_folderN/create_parent_folderlogging_remote_workN/logging_remote_workrun_configurationPentaho local/run_configurationparameterspass_all_parametersY/pass_all_parameters/parametersparallelN/paralleldrawY/drawnr0/nrxloc640/xlocyloc288/yloc/entry/entrieshopshopfromSTART/fromto初始化所有表/tofrom_nr0/from_nrto_nr0/to_nrenabledY/enabledevaluationY/evaluationunconditionalY/unconditional/hophopfrom初始化所有表/fromto同步aav/tofrom_nr0/from_nrto_nr0/to_nrenabledY/enabledevaluationY/evaluationunconditionalN/unconditional/hophopfrom同步aav/fromtoDUMMY/tofrom_nr0/from_nrto_nr0/to_nrenabledY/enabledevaluationY/evaluationunconditionalN/unconditional/hophopfrom初始化所有表/fromto同步douban/tofrom_nr0/from_nrto_nr0/to_nrenabledY/enabledevaluationY/evaluationunconditionalN/unconditional/hophopfrom初始化所有表/fromto同步lottery/tofrom_nr0/from_nrto_nr0/to_nrenabledY/enabledevaluationY/evaluationunconditionalN/unconditional/hophopfrom初始化所有表/fromto同步sp-dev/tofrom_nr0/from_nrto_nr0/to_nrenabledY/enabledevaluationY/evaluationunconditionalN/unconditional/hophopfrom初始化所有表/fromto同步aps/tofrom_nr0/from_nrto_nr0/to_nrenabledY/enabledevaluationY/evaluationunconditionalN/unconditional/hophopfrom同步aps/fromtoDUMMY/tofrom_nr0/from_nrto_nr0/to_nrenabledY/enabledevaluationY/evaluationunconditionalN/unconditional/hophopfrom同步douban/fromtoDUMMY/tofrom_nr0/from_nrto_nr0/to_nrenabledY/enabledevaluationY/evaluationunconditionalN/unconditional/hophopfrom同步lottery/fromtoDUMMY/tofrom_nr0/from_nrto_nr0/to_nrenabledY/enabledevaluationY/evaluationunconditionalN/unconditional/hophopfrom同步sp-dev/fromtoDUMMY/tofrom_nr0/from_nrto_nr0/to_nrenabledY/enabledevaluationY/evaluationunconditionalN/unconditional/hop/hopsnotepads/notepadsattributesgroupnameMETASTORE.pentaho/nameattributekeyDefault Run Configuration/keyvalue{namespace:pentaho,id:Default Run Configuration,name:Default Run Configuration,description:Defines a default run configuration,metaStoreName:null}/value/attribute/groupgroupname{_:Embedded MetaStore Elements,namespace:pentaho,type:Default Run Configuration}/nameattributekeyPentaho local/keyvalue{children:[{children:[],id:server,value:null},{children:[],id:clustered,value:N},{children:[],id:name,value:Pentaho local},{children:[],id:description,value:null},{children:[],id:readOnly,value:Y},{children:[],id:sendResources,value:N},{children:[],id:logRemoteExecutionLocally,value:N},{children:[],id:remote,value:N},{children:[],id:local,value:Y},{children:[],id:showTransformations,value:N}],id:Pentaho local,value:null,name:Pentaho local,owner:null,ownerPermissionsList:[]}/value/attribute/group/attributes
/job4kettle.properties变量
名称 值
local_ip localhost
local_user root
password 123456
schemeName_aav aav
schemeName_aav1 aav1
schemeName_aps aps
schemeName_aps1 aps1
schemeName_douban douban
schemeName_douban1 douban1
schemeName_lottery lottery
schemeName_lottery1 lottery1
schemeName_sp-dev sp-dev
schemeName_sp-dev1 sp-dev15在使用组件时经常会遇到一个问题数据库中某个值设置的是非null在同步数据时起始源数据库的表字段值为空字符串在进入到kettle中时会默认编程null这时直接同步会报错需要更改kettle的配置变量。 kettle.properties -》KETTLE_EMPTY_STRING_DIFFERS_FROM_NULLY
更改后需要重启kettle
