当前位置：前端开发 >  【用户投稿】Apache SeaTunnel 2.3.3+Web 1.0.0版本安装部署

【用户投稿】Apache SeaTunnel 2.3.3+Web 1.0.0版本安装部署

SeaTunnel 2024-08-02 12:03:01 阅读 71

项目概要

Apache SeaTunnel 是一个分布式、高性能、易扩展的数据集成平台，用于实时和离线数据处理,支持多种数据源之间的数据迁移和转换。

其中，<code>Apache-seatunnel-web-1.0.0-bin.tar.gz和apache-seatunnel-2.3.3-bin.tar.gz代表了 Apache SeaTunnel Web 界面的安装包和 Apache SeaTunnel 的核心引擎的安装包。

**apache-seatunnel-web-1.0.0-bin.tar.gz：**这是 Apache SeaTunnel Web 的安装包，提供了用户友好的 Web 界面，使得用户可以更方便地管理和监控 SeaTunnel 任务，包括创建、编辑和运行作业等操作。通过这个界面，用户可以无需直接接触命令行就能进行配置和管理。**apache-seatunnel-2.3.3-bin.tar.gz：**这是 Apache SeaTunnel 核心引擎的二进制发行版，包含了执行数据抽取、转换和加载的核心代码库。它是实际执行数据处理任务的部分，可以与各种数据源和目标（如数据库、文件系统、消息队列等）进行交互。

在部署 Apache SeaTunnel 时，通常需要先安装并配置好核心引擎，然后根据需要选择是否使用 Web 界面进行管理。如果你只需要在命令行下运行和管理任务，那么只需要安装核心引擎即可。如果你想拥有一个图形化的任务管理界面，那么就需要同时安装 Apache SeaTunnel Web。

file

安装包及源码包下载地址：https://seatunnel.apache.org/download/

file

在部署 Apache SeaTunnel 时，通常需要先安装并配置好核心引擎，然后根据需要选择是否使用 Web 界面进行管理。如果你只需要在命令行下运行和管理任务，那么只需要安装核心引擎即可。如果你想拥有一个图形化的任务管理界面，那么就需要同时安装 Apache SeaTunnel Web。

核心引擎部署

上传解压安装包： tar xf apache-seatunnel-2.3.3-bin.tar.gz

file

自动安装

自动安装插件（全量安装，速度比较慢，一直到下班，一个半小时都没结束，第二天早上来显示下载结束）

<code>[opensource@bigdata02 bin]$ pwd

/home/opensource/app/apache-seatunnel-2.3.3/bin

[opensource@bigdata02 bin]$ ls

install-plugin.sh seatunnel.sh start-seatunnel-flink-15-connector-v2.sh start-seatunnel-spark-3-connector-v2.sh

seatunnel-cluster.sh start-seatunnel-flink-13-connector-v2.sh start-seatunnel-spark-2-connector-v2.sh stop-seatunnel-cluster.sh

[opensource@bigdata02 bin]$

[opensource@bigdata02 bin]$ ./install-plugin.sh

file

验证服务

执行系统自带的任务进行验证：<code> sh seatunnel.sh --config ../config/v2.batch.config.template -m local

file

<code>[opensource@bigdata02 bin]$ ls

install-plugin.sh seatunnel.sh start-seatunnel-flink-15-connector-v2.sh start-seatunnel-spark-3-connector-v2.sh

seatunnel-cluster.sh start-seatunnel-flink-13-connector-v2.sh start-seatunnel-spark-2-connector-v2.sh stop-seatunnel-cluster.sh

[opensource@bigdata02 bin]$

[opensource@bigdata02 bin]$ nohup sh seatunnel-cluster.sh 2>&1 &

[1] 93523

[opensource@bigdata02 bin]$ nohup: 忽略输入并把输出追加到"nohup.out"

[opensource@bigdata02 bin]$ ls

install-plugin.sh seatunnel-cluster.sh start-seatunnel-flink-13-connector-v2.sh start-seatunnel-spark-2-connector-v2.sh stop-seatunnel-cluster.sh

nohup.out seatunnel.sh start-seatunnel-flink-15-connector-v2.sh start-seatunnel-spark-3-connector-v2.sh

[opensource@bigdata02 bin]$ tail -f nohup.out

十二月 08, 2023 9:30:58 上午 com.hazelcast.internal.config.AbstractConfigLocator

信息: Using configuration file at /home/opensource/app/apache-seatunnel-2.3.3/config/hazelcast.yaml

十二月 08, 2023 9:30:59 上午 com.hazelcast.internal.config.AbstractConfigLocator

信息: Loading configuration '/home/opensource/app/apache-seatunnel-2.3.3/config/seatunnel.yaml' from System property 'seatunnel.config'

十二月 08, 2023 9:30:59 上午 com.hazelcast.internal.config.AbstractConfigLocator

信息: Using configuration file at /home/opensource/app/apache-seatunnel-2.3.3/config/seatunnel.yaml

十二月 08, 2023 9:30:59 上午 com.hazelcast.internal.config.AbstractConfigLocator

信息: Loading configuration '/home/opensource/app/apache-seatunnel-2.3.3/config/hazelcast.yaml' from System property 'hazelcast.config'

十二月 08, 2023 9:30:59 上午 com.hazelcast.internal.config.AbstractConfigLocator

信息: Using configuration file at /home/opensource/app/apache-seatunnel-2.3.3/config/hazelcast.yaml

^C

[opensource@bigdata02 bin]$ tail -f ../logs/seatunnel-engine-server.log

Member [localhost]:5801 - 9db58520-2335-441c-8999-ad78123f6509 this

]

2023-12-08 09:30:59,930 INFO com.hazelcast.core.LifecycleService - [localhost]:5801 [seatunnel] [5.1] [localhost]:5801 is STARTED

2023-12-08 09:31:01,865 INFO org.apache.seatunnel.engine.server.CoordinatorService - [localhost]:5801 [seatunnel] [5.1] This node become a new active master node, begin init coordinator service

2023-12-08 09:31:02,061 INFO com.hazelcast.internal.partition.impl.PartitionStateManager - [localhost]:5801 [seatunnel] [5.1] Initializing cluster partition table arrangement...

2023-12-08 09:31:04,863 INFO org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - Init ResourceManager

2023-12-08 09:31:04,863 INFO org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - initWorker...

2023-12-08 09:31:04,864 INFO org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - initWorker live nodes: [[localhost]:5801]

2023-12-08 09:31:04,865 INFO org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - registerWorker: {[localhost]:5801=WorkerProfile(address=[localhost]:5801, profile=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=2058354688}}, unassignedResource=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=2058354688}}, assignedSlots=[], unassignedSlots=[])}

2023-12-08 09:31:59,863 INFO org.apache.seatunnel.engine.server.CoordinatorService - [localhost]:5801 [seatunnel] [5.1]

***********************************************

CoordinatorService Thread Pool Status

***********************************************

activeCount : 0

corePoolSize : 0

maximumPoolSize : 2147483647

poolSize : 0

completedTaskCount : 0

taskCount : 0

***********************************************

2023-12-08 09:31:59,867 INFO org.apache.seatunnel.engine.server.CoordinatorService - [localhost]:5801 [seatunnel] [5.1]

***********************************************

Job info detail

***********************************************

createdJobCount : 0

scheduledJobCount : 0

runningJobCount : 0

failingJobCount : 0

failedJobCount : 0

cancellingJobCount : 0

canceledJobCount : 0

finishedJobCount : 0

restartingJobCount : 0

suspendedJobCount : 0

reconcilingJobCount : 0

***********************************************

启动服务

启动服务并且后台运行，用于web端连接：nohup sh seatunnel-cluster.sh 2>&1 &

file

<code>[opensource@bigdata02 bin]$ ls

install-plugin.sh seatunnel.sh start-seatunnel-flink-15-connector-v2.sh start-seatunnel-spark-3-connector-v2.sh

seatunnel-cluster.sh start-seatunnel-flink-13-connector-v2.sh start-seatunnel-spark-2-connector-v2.sh stop-seatunnel-cluster.sh

[opensource@bigdata02 bin]$

[opensource@bigdata02 bin]$ nohup sh seatunnel-cluster.sh 2>&1 &

[1] 93523

[opensource@bigdata02 bin]$ nohup: 忽略输入并把输出追加到"nohup.out"

[opensource@bigdata02 bin]$ ls

install-plugin.sh seatunnel-cluster.sh start-seatunnel-flink-13-connector-v2.sh start-seatunnel-spark-2-connector-v2.sh stop-seatunnel-cluster.sh

nohup.out seatunnel.sh start-seatunnel-flink-15-connector-v2.sh start-seatunnel-spark-3-connector-v2.sh

[opensource@bigdata02 bin]$ tail -f nohup.out

十二月 08, 2023 9:30:58 上午 com.hazelcast.internal.config.AbstractConfigLocator

信息: Using configuration file at /home/opensource/app/apache-seatunnel-2.3.3/config/hazelcast.yaml

十二月 08, 2023 9:30:59 上午 com.hazelcast.internal.config.AbstractConfigLocator

信息: Loading configuration '/home/opensource/app/apache-seatunnel-2.3.3/config/seatunnel.yaml' from System property 'seatunnel.config'

十二月 08, 2023 9:30:59 上午 com.hazelcast.internal.config.AbstractConfigLocator

信息: Using configuration file at /home/opensource/app/apache-seatunnel-2.3.3/config/seatunnel.yaml

十二月 08, 2023 9:30:59 上午 com.hazelcast.internal.config.AbstractConfigLocator

信息: Loading configuration '/home/opensource/app/apache-seatunnel-2.3.3/config/hazelcast.yaml' from System property 'hazelcast.config'

十二月 08, 2023 9:30:59 上午 com.hazelcast.internal.config.AbstractConfigLocator

信息: Using configuration file at /home/opensource/app/apache-seatunnel-2.3.3/config/hazelcast.yaml

^C

[opensource@bigdata02 bin]$ tail -f ../logs/seatunnel-engine-server.log

Member [localhost]:5801 - 9db58520-2335-441c-8999-ad78123f6509 this

]

2023-12-08 09:30:59,930 INFO com.hazelcast.core.LifecycleService - [localhost]:5801 [seatunnel] [5.1] [localhost]:5801 is STARTED

2023-12-08 09:31:01,865 INFO org.apache.seatunnel.engine.server.CoordinatorService - [localhost]:5801 [seatunnel] [5.1] This node become a new active master node, begin init coordinator service

2023-12-08 09:31:02,061 INFO com.hazelcast.internal.partition.impl.PartitionStateManager - [localhost]:5801 [seatunnel] [5.1] Initializing cluster partition table arrangement...

2023-12-08 09:31:04,863 INFO org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - Init ResourceManager

2023-12-08 09:31:04,863 INFO org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - initWorker...

2023-12-08 09:31:04,864 INFO org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - initWorker live nodes: [[localhost]:5801]

2023-12-08 09:31:04,865 INFO org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - registerWorker: {[localhost]:5801=WorkerProfile(address=[localhost]:5801, profile=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=2058354688}}, unassignedResource=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=2058354688}}, assignedSlots=[], unassignedSlots=[])}

2023-12-08 09:31:59,863 INFO org.apache.seatunnel.engine.server.CoordinatorService - [localhost]:5801 [seatunnel] [5.1]

***********************************************

CoordinatorService Thread Pool Status

***********************************************

activeCount : 0

corePoolSize : 0

maximumPoolSize : 2147483647

poolSize : 0

completedTaskCount : 0

taskCount : 0

***********************************************

2023-12-08 09:31:59,867 INFO org.apache.seatunnel.engine.server.CoordinatorService - [localhost]:5801 [seatunnel] [5.1]

***********************************************

Job info detail

***********************************************

createdJobCount : 0

scheduledJobCount : 0

runningJobCount : 0

failingJobCount : 0

failedJobCount : 0

cancellingJobCount : 0

canceledJobCount : 0

finishedJobCount : 0

restartingJobCount : 0

suspendedJobCount : 0

reconcilingJobCount : 0

***********************************************

手动安装

不想全部安装，修改配置：vi config/plugin_config

file

无法在线安装，只能手动安装，maven库下载地址：https://repo.maven.apache.org/maven2/org/apache/seatunnel/

file

对于自动下载下来的jar包，已经全部打包上传到CSDN,大家可以免费下载，直接放到connectors/seatunnel

file

下载地址：https://download.csdn.net/download/qq_36434219/88609069

file

CSDN资源大小不允许超过1000M,这里采用.tar.xz格式（解压会有些慢，小十分钟吧）

三种压缩命令：

<code>tar -cf seatunnel_connectors_2.3.3.tar seatunnel

tar -czf seatunnel_connectors_2.3.3.tar.gz seatunnel

tar -cJf seatunnel_connectors_2.3.3.tar.xz seatunnel

对应解压命令：

tar -xf seatunnel_connectors_2.3.3.tar

tar -xzf seatunnel_connectors_2.3.3.tar.gz

tar -xJf seatunnel_connectors_2.3.3.tar.xz

file

Web界面部署

上传解压web部署包：<code>tar xf apache-seatunnel-web-1.0.0-bin.tar.gz

初始化数据库

初始化SQL脚本 script/seatunnel_server_mysql.sql

file

file

<code>SET FOREIGN_KEY_CHECKS = 0;

-- ----------------------------

-- Table structure for role

-- ----------------------------

DROP TABLE IF EXISTS `role`;

CREATE TABLE `role` (

`id` int(20) NOT NULL AUTO_INCREMENT,

`type` int(2) NOT NULL,

`role_name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL,

`description` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NULL DEFAULT NULL,

`create_time` datetime(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),

`update_time` datetime(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),

PRIMARY KEY (`id`) USING BTREE

) ENGINE = InnoDB AUTO_INCREMENT = 1 CHARACTER SET = utf8mb4 COLLATE = utf8mb4_general_ci ROW_FORMAT = Dynamic;

-- ----------------------------

-- Records of role

-- ----------------------------

INSERT INTO `role`(`type`,`role_name`,`description`) values (0, 'ADMIN_ROLE', 'Admin User');

INSERT INTO `role`(`type`,`role_name`,`description`) values (1, 'NORMAL_ROLE', 'Normal User');

-- ----------------------------

-- Table structure for role_user_relation

-- ----------------------------

DROP TABLE IF EXISTS `role_user_relation`;

CREATE TABLE `role_user_relation` (

`id` int(20) NOT NULL AUTO_INCREMENT,

`role_id` int(20) NOT NULL,

`user_id` int(20) NOT NULL,

`create_time` datetime(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),

`update_time` datetime(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),

PRIMARY KEY (`id`) USING BTREE

) ENGINE = InnoDB AUTO_INCREMENT = 1 CHARACTER SET = utf8mb4 COLLATE = utf8mb4_general_ci ROW_FORMAT = Dynamic;

-- ----------------------------

-- Table structure for t_st_datasource

-- ----------------------------

DROP TABLE IF EXISTS `t_st_datasource`;

CREATE TABLE `t_st_datasource` (

`id` bigint(20) NOT NULL,

`datasource_name` varchar(63) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,

`plugin_name` varchar(63) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,

`plugin_version` varchar(63) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT '1.0.0',

`datasource_config` varchar(1023) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,

`description` varchar(63) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,

`create_user_id` int(11) NOT NULL,

`update_user_id` int(11) NOT NULL,

`create_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),

`update_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),

PRIMARY KEY (`id`) USING BTREE,

UNIQUE INDEX `t_st_datasource_datasource_name_uindex`(`datasource_name`) USING BTREE

) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Dynamic;

-- ----------------------------

-- Table structure for t_st_job_definition

-- ----------------------------

DROP TABLE IF EXISTS `t_st_job_definition`;

CREATE TABLE `t_st_job_definition` (

`id` bigint(20) NOT NULL,

`name` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,

`description` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,

`job_type` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,

`create_user_id` int(11) NOT NULL,

`update_user_id` int(11) NOT NULL,

`create_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),

`update_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),

PRIMARY KEY (`id`) USING BTREE,

UNIQUE INDEX `name`(`name`) USING BTREE

) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Dynamic;

-- ----------------------------

-- Table structure for t_st_job_instance

-- ----------------------------

DROP TABLE IF EXISTS `t_st_job_instance`;

CREATE TABLE `t_st_job_instance` (

`id` bigint(20) NOT NULL,

`job_define_id` bigint(20) NOT NULL,

`job_status` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,

`job_config` text CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,

`engine_name` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,

`engine_version` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,

`job_engine_id` varchar(200) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,

`create_user_id` int(20) NOT NULL,

`update_user_id` int(20) NULL DEFAULT NULL,

`create_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),

`update_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),

`end_time` timestamp(3) NULL DEFAULT NULL,

`job_type` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,

PRIMARY KEY (`id`) USING BTREE

) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Dynamic;

-- ----------------------------

-- Table structure for t_st_job_instance_history

-- ----------------------------

DROP TABLE IF EXISTS `t_st_job_instance_history`;

CREATE TABLE `t_st_job_instance_history` (

`id` bigint(20) NOT NULL,

`dag` text CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,

`create_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),

`update_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),

PRIMARY KEY (`id`) USING BTREE

) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Dynamic;

-- ----------------------------

-- Table structure for t_st_job_line

-- ----------------------------

DROP TABLE IF EXISTS `t_st_job_line`;

CREATE TABLE `t_st_job_line` (

`id` bigint(20) NOT NULL,

`version_id` bigint(20) NOT NULL,

`input_plugin_id` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,

`target_plugin_id` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,

`create_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),

`update_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),

PRIMARY KEY (`id`) USING BTREE,

INDEX `job_line_version_index`(`version_id`) USING BTREE

) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Dynamic;

-- ----------------------------

-- Table structure for t_st_job_metrics

-- ----------------------------

DROP TABLE IF EXISTS `t_st_job_metrics`;

CREATE TABLE `t_st_job_metrics` (

`id` bigint(20) NOT NULL,

`job_instance_id` bigint(20) NOT NULL,

`pipeline_id` int(20) NOT NULL,

`read_row_count` bigint(20) NOT NULL,

`write_row_count` bigint(20) NOT NULL,

`source_table_names` varchar(200) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,

`sink_table_names` varchar(200) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,

`read_qps` bigint(20) NULL DEFAULT NULL,

`write_qps` bigint(20) NULL DEFAULT NULL,

`record_delay` bigint(20) NULL DEFAULT NULL,

`status` varchar(20) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,

`create_user_id` int(20) NOT NULL,

`update_user_id` int(20) NULL DEFAULT NULL,

`create_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),

`update_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),

PRIMARY KEY (`id`) USING BTREE

) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Dynamic;

-- ----------------------------

-- Table structure for t_st_job_task

-- ----------------------------

DROP TABLE IF EXISTS `t_st_job_task`;

CREATE TABLE `t_st_job_task` (

`id` bigint(20) NOT NULL,

`version_id` bigint(20) NOT NULL,

`plugin_id` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,

`name` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,

`config` text CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL,

`transform_options` varchar(5000) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,

`output_schema` text CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL,

`connector_type` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,

`datasource_id` bigint(20) NULL DEFAULT NULL,

`datasource_option` varchar(5000) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,

`select_table_fields` varchar(5000) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,

`scene_mode` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,

`type` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,

`create_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),

`update_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),

PRIMARY KEY (`id`) USING BTREE,

INDEX `job_task_plugin_id_index`(`plugin_id`) USING BTREE

) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Dynamic;

-- ----------------------------

-- Table structure for t_st_job_version

-- ----------------------------

DROP TABLE IF EXISTS `t_st_job_version`;

CREATE TABLE `t_st_job_version` (

`id` bigint(20) NOT NULL,

`job_id` bigint(20) NOT NULL,

`name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,

`job_mode` varchar(10) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,

`env` text CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL,

`engine_name` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,

`engine_version` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,

`create_user_id` int(11) NOT NULL,

`update_user_id` int(11) NOT NULL,

`create_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),

`update_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),

PRIMARY KEY (`id`) USING BTREE

) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Dynamic;

-- ----------------------------

-- Table structure for t_st_virtual_table

-- ----------------------------

DROP TABLE IF EXISTS `t_st_virtual_table`;

CREATE TABLE `t_st_virtual_table` (

`id` bigint(20) NOT NULL,

`datasource_id` bigint(20) NOT NULL,

`virtual_database_name` varchar(63) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,

`virtual_table_name` varchar(63) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,

`table_fields` varchar(1023) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,

`virtual_table_config` varchar(1023) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,

`description` varchar(63) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,

`create_user_id` int(11) NOT NULL,

`update_user_id` int(11) NOT NULL,

`create_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),

`update_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),

PRIMARY KEY (`id`) USING BTREE

) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Dynamic;

-- ----------------------------

-- Table structure for user

-- ----------------------------

DROP TABLE IF EXISTS `user`;

CREATE TABLE `user` (

`id` int(11) NOT NULL AUTO_INCREMENT,

`username` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL,

`password` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL,

`status` tinyint(4) NOT NULL,

`type` tinyint(4) NOT NULL,

`create_time` datetime(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),

`update_time` datetime(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),

PRIMARY KEY (`id`) USING BTREE

) ENGINE = InnoDB AUTO_INCREMENT = 2 CHARACTER SET = utf8mb4 COLLATE = utf8mb4_general_ci ROW_FORMAT = Dynamic;

-- ----------------------------

-- Table structure for user_login_log

-- ----------------------------

DROP TABLE IF EXISTS `user_login_log`;

CREATE TABLE `user_login_log` (

`id` bigint(20) NOT NULL AUTO_INCREMENT,

`user_id` int(11) NOT NULL,

`token` mediumtext CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL,

`token_status` tinyint(1) NOT NULL,

`create_time` datetime(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),

`update_time` datetime(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),

PRIMARY KEY (`id`) USING BTREE

) ENGINE = InnoDB AUTO_INCREMENT = 106 CHARACTER SET = utf8mb4 COLLATE = utf8mb4_general_ci ROW_FORMAT = Dynamic;

-- ----------------------------

-- Records of user_login_log

-- ----------------------------

INSERT INTO `user`(`username`,`password`,`status`,`type`) values ('admin', '7f97da8846fed829bb8d1fd9f8030f3b', 0, 0);

SET FOREIGN_KEY_CHECKS = 1;

配置文件修改

配置数据库信息： vi conf/application.yml

file

复制引擎服务中配置文件到Web配置目录下面

<code>[opensource@bigdata02 app]$ cp apache-seatunnel-2.3.3/config/hazelcast-client.yaml apache-seatunnel-web-1.0.0-bin/conf/

[opensource@bigdata02 app]$ cp apache-seatunnel-2.3.3/connectors/plugin-mapping.properties apache-seatunnel-web-1.0.0-bin/conf/

file

配置下载数据源jar包

选择的Mysql作为元数据库，对应的驱动包<code>mysql-connector-java-8.0.25.jar放到libs下

file

下载其它数据源资源包,首先下载源码，然后执行下载脚本<code>download_datasource.sh(本地安装过git)

源码地址:https://github.com/apache/seatunnel-web

file

修改<code>download_datasource.sh相关配置

file

file

执行<code>download_datasource.sh

file

开始下载jar包

file

下载完成

file

上传到libs目录

file

启动服务

<code>sh bin/seatunnel-backend-daemon.sh start

file

Web页面一览

登录

登录地址：http://xxxx:8801/ui/#/login ;

用户密码 admin/admin

file

file

数据源

file

file

任务

file

尝试创建整库同步

file

用户管理

file

虚拟表

file

其他

问题记录

服务启动失败：Application run failed org.yaml.snakeyaml.scanner.ScannerException: while scanning an anchor

<code>14:29:08.032 [main] ERROR org.springframework.boot.SpringApplication - Application run failed

org.yaml.snakeyaml.scanner.ScannerException: while scanning an anchor

in 'reader', line 30, column 15:

password: &R7#J*vDVWEuT48m

^

unexpected character found *(42)

in 'reader', line 30, column 20:

password: &R7#J*vDVWEuT48m

^

at org.yaml.snakeyaml.scanner.ScannerImpl.scanAnchor(ScannerImpl.java:1507)

at org.yaml.snakeyaml.scanner.ScannerImpl.fetchAnchor(ScannerImpl.java:958)

at org.yaml.snakeyaml.scanner.ScannerImpl.fetchMoreTokens(ScannerImpl.java:389)

at org.yaml.snakeyaml.scanner.ScannerImpl.checkToken(ScannerImpl.java:248)

at org.yaml.snakeyaml.parser.ParserImpl$ParseBlockMappingValue.produce(ParserImpl.java:665)

at org.yaml.snakeyaml.parser.ParserImpl.peekEvent(ParserImpl.java:165)

at org.yaml.snakeyaml.comments.CommentEventsCollector$1.peek(CommentEventsCollector.java:59)

at org.yaml.snakeyaml.comments.CommentEventsCollector$1.peek(CommentEventsCollector.java:45)

at org.yaml.snakeyaml.comments.CommentEventsCollector.collectEvents(CommentEventsCollector.java:140)

at org.yaml.snakeyaml.comments.CommentEventsCollector.collectEvents(CommentEventsCollector.java:119)

at org.yaml.snakeyaml.composer.Composer.composeScalarN

解决办法： 密码&R7#J*vDVWEuT48m增加双引号

file

file

原文链接：https://blog.csdn.net/qq_36434219/article/details/134857560

本文由 白鲸开源科技 提供发布支持！

上一篇： 前端：Nuxt3 + Vuetify3 + Element Plus + 添加常用插件

下一篇： 【UE5】【插件】【WebUI】内置浏览器插件WebUI简易使用笔记

本文标签

【用户投稿】Apache SeaTunnel 2.3.3+Web 1.0.0版本安装部署   

声明

本文内容仅代表作者观点，或转载于其他网站，本站不以此文作为商业用途
如有涉及侵权，请联系本站进行删除
转载本站原创文章，请注明来源及作者。

相关文章

阅读排行

热门文章