轻松实现数据抽取工具kettle与阿里云大数据maxcompute交互

插件部署

下载MaxCompute的Kettle插件包

$ wget http://repo.aliyun.com/download/aliyun-kettle-odps-plugin-1.0.0.tar.gz

插件解压

这里以kettle7.0版本为例,

解压文件aliyun-kettle-odps-plugin-1.0.0.tar.gz到目录plugins下:

以下是对应的具体目录pdi-ce-7.0.0.0-25\data-integration\plugins

打开kettle

打开kettle软件检测maxcompute插件是否安装成功:

https://blog.juncdt.com/wp-content/uploads/2018/07/1.png

注:由上表示已把maxcompute插件导入到kettle数据抽取工具中

在maxcompute客户端查看,t2表中有两条数据,具体操作如下次:

odps@ gap>select * from t2;

ID = 20180702064042887gzcin8jc2

Log view:

http://logview.odps.aliyun.com/logview/?h=http://service.odps.aliyun.com/api&p=g

ap&i=20180702064042887gzcin8jc2&token=OUc3a0dLVUNjUytvdFh5dURpa1g1Ymw1OW5NPSxPRF

BTX09CTzpwNF8yMDg5MzQ0MjgyNDc0MDM3NzAsMTUzMTExODQ0Myx7IlN0YXRlbWVudCI6W3siQWN0aW

9uIjpbIm9kcHM6UmVhZCJdLCJFZmZlY3QiOiJBbGxvdyIsIlJlc291cmNlIjpbImFjczpvZHBzOio6cH

JvamVjdHMvZ2FwL2luc3RhbmNlcy8yMDE4MDcwMjA2NDA0Mjg4N2d6Y2luOGpjMiJdfV0sIlZlcnNpb2

4iOiIxIn0= Job Queueing... +------------+------------+

| id | name | +------------+------------+

| 3 | uuf | +------------+------------+

1 records (at most 10000 supported) fetched by instance tunnel.

kettle实现maxcompute与本地mysql数据同步

odps数据输入配置

查看配置:

https://blog.juncdt.com/wp-content/uploads/2018/07/2.png

数据插入

数据插入操作:

https://blog.juncdt.com/wp-content/uploads/2018/07/3.png

整个数据操作流程

整个流程如下次:

https://blog.juncdt.com/wp-content/uploads/2018/07/4.png

注:数据导入成功后,mysql表t2中也生成一条数据:

select * from t2;

+------------+------------+

| id | name |

+------------+------------+

| 3 | uuf |

+------------+------------+

1 records (at most 10000 supported) fetched by instance tunnel.

发表评论

发表评论

*

沙发空缺中,还不快抢~