【用户投稿】Apache SeaTunnel 2.3.3+Web 1.0.0版本安装部署
SeaTunnel 2024-08-02 12:03:01 阅读 71
项目概要
Apache SeaTunnel 是一个分布式、高性能、易扩展的数据集成平台，用于实时和离线数据处理,支持多种数据源之间的数据迁移和转换。
其中，<code>Apache-seatunnel-web-1.0.0-bin.tar.gz和apache-seatunnel-2.3.3-bin.tar.gz代表了 Apache SeaTunnel Web 界面的安装包和 Apache SeaTunnel 的核心引擎的安装包。
**apache-seatunnel-web-1.0.0-bin.tar.gz：**这是 Apache SeaTunnel Web 的安装包，提供了用户友好的 Web 界面，使得用户可以更方便地管理和监控 SeaTunnel 任务，包括创建、编辑和运行作业等操作。通过这个界面，用户可以无需直接接触命令行就能进行配置和管理。**apache-seatunnel-2.3.3-bin.tar.gz：**这是 Apache SeaTunnel 核心引擎的二进制发行版，包含了执行数据抽取、转换和加载的核心代码库。它是实际执行数据处理任务的部分，可以与各种数据源和目标（如数据库、文件系统、消息队列等）进行交互。
在部署 Apache SeaTunnel 时，通常需要先安装并配置好核心引擎，然后根据需要选择是否使用 Web 界面进行管理。如果你只需要在命令行下运行和管理任务，那么只需要安装核心引擎即可。如果你想拥有一个图形化的任务管理界面，那么就需要同时安装 Apache SeaTunnel Web。
安装包及源码包下载地址：https://seatunnel.apache.org/download/
在部署 Apache SeaTunnel 时，通常需要先安装并配置好核心引擎，然后根据需要选择是否使用 Web 界面进行管理。如果你只需要在命令行下运行和管理任务，那么只需要安装核心引擎即可。如果你想拥有一个图形化的任务管理界面，那么就需要同时安装 Apache SeaTunnel Web。
核心引擎部署
上传解压安装包： tar xf apache-seatunnel-2.3.3-bin.tar.gz
自动安装
自动安装插件（全量安装，速度比较慢，一直到下班，一个半小时都没结束，第二天早上来显示下载结束）
<code>[opensource@bigdata02 bin]$ pwd
/home/opensource/app/apache-seatunnel-2.3.3/bin
[opensource@bigdata02 bin]$ ls
install-plugin.sh seatunnel.sh start-seatunnel-flink-15-connector-v2.sh start-seatunnel-spark-3-connector-v2.sh
seatunnel-cluster.sh start-seatunnel-flink-13-connector-v2.sh start-seatunnel-spark-2-connector-v2.sh stop-seatunnel-cluster.sh
[opensource@bigdata02 bin]$
[opensource@bigdata02 bin]$ ./install-plugin.sh
验证服务
执行系统自带的任务进行验证：<code> sh seatunnel.sh --config ../config/v2.batch.config.template -m local
<code>[opensource@bigdata02 bin]$ ls
install-plugin.sh seatunnel.sh start-seatunnel-flink-15-connector-v2.sh start-seatunnel-spark-3-connector-v2.sh
seatunnel-cluster.sh start-seatunnel-flink-13-connector-v2.sh start-seatunnel-spark-2-connector-v2.sh stop-seatunnel-cluster.sh
[opensource@bigdata02 bin]$
[opensource@bigdata02 bin]$ nohup sh seatunnel-cluster.sh 2>&1 &
[1] 93523
[opensource@bigdata02 bin]$ nohup: 忽略输入并把输出追加到"nohup.out"
[opensource@bigdata02 bin]$ ls
install-plugin.sh seatunnel-cluster.sh start-seatunnel-flink-13-connector-v2.sh start-seatunnel-spark-2-connector-v2.sh stop-seatunnel-cluster.sh
nohup.out seatunnel.sh start-seatunnel-flink-15-connector-v2.sh start-seatunnel-spark-3-connector-v2.sh
[opensource@bigdata02 bin]$ tail -f nohup.out
十二月 08, 2023 9:30:58 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Using configuration file at /home/opensource/app/apache-seatunnel-2.3.3/config/hazelcast.yaml
十二月 08, 2023 9:30:59 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Loading configuration '/home/opensource/app/apache-seatunnel-2.3.3/config/seatunnel.yaml' from System property 'seatunnel.config'
十二月 08, 2023 9:30:59 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Using configuration file at /home/opensource/app/apache-seatunnel-2.3.3/config/seatunnel.yaml
十二月 08, 2023 9:30:59 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Loading configuration '/home/opensource/app/apache-seatunnel-2.3.3/config/hazelcast.yaml' from System property 'hazelcast.config'
十二月 08, 2023 9:30:59 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Using configuration file at /home/opensource/app/apache-seatunnel-2.3.3/config/hazelcast.yaml
^C
[opensource@bigdata02 bin]$ tail -f ../logs/seatunnel-engine-server.log
Member [localhost]:5801 - 9db58520-2335-441c-8999-ad78123f6509 this
]
2023-12-08 09:30:59,930 INFO com.hazelcast.core.LifecycleService - [localhost]:5801 [seatunnel] [5.1] [localhost]:5801 is STARTED
2023-12-08 09:31:01,865 INFO org.apache.seatunnel.engine.server.CoordinatorService - [localhost]:5801 [seatunnel] [5.1] This node become a new active master node, begin init coordinator service
2023-12-08 09:31:02,061 INFO com.hazelcast.internal.partition.impl.PartitionStateManager - [localhost]:5801 [seatunnel] [5.1] Initializing cluster partition table arrangement...
2023-12-08 09:31:04,863 INFO org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - Init ResourceManager
2023-12-08 09:31:04,863 INFO org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - initWorker...
2023-12-08 09:31:04,864 INFO org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - initWorker live nodes: [[localhost]:5801]
2023-12-08 09:31:04,865 INFO org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - registerWorker: {[localhost]:5801=WorkerProfile(address=[localhost]:5801, profile=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=2058354688}}, unassignedResource=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=2058354688}}, assignedSlots=[], unassignedSlots=[])}
2023-12-08 09:31:59,863 INFO org.apache.seatunnel.engine.server.CoordinatorService - [localhost]:5801 [seatunnel] [5.1]
***********************************************
CoordinatorService Thread Pool Status
***********************************************
activeCount : 0
corePoolSize : 0
maximumPoolSize : 2147483647
poolSize : 0
completedTaskCount : 0
taskCount : 0
***********************************************
2023-12-08 09:31:59,867 INFO org.apache.seatunnel.engine.server.CoordinatorService - [localhost]:5801 [seatunnel] [5.1]
***********************************************
Job info detail
***********************************************
createdJobCount : 0
scheduledJobCount : 0
runningJobCount : 0
failingJobCount : 0
failedJobCount : 0
cancellingJobCount : 0
canceledJobCount : 0
finishedJobCount : 0
restartingJobCount : 0
suspendedJobCount : 0
reconcilingJobCount : 0
***********************************************
启动服务
启动服务并且后台运行，用于web端连接：nohup sh seatunnel-cluster.sh 2>&1 &
<code>[opensource@bigdata02 bin]$ ls
install-plugin.sh seatunnel.sh start-seatunnel-flink-15-connector-v2.sh start-seatunnel-spark-3-connector-v2.sh
seatunnel-cluster.sh start-seatunnel-flink-13-connector-v2.sh start-seatunnel-spark-2-connector-v2.sh stop-seatunnel-cluster.sh
[opensource@bigdata02 bin]$
[opensource@bigdata02 bin]$ nohup sh seatunnel-cluster.sh 2>&1 &
[1] 93523
[opensource@bigdata02 bin]$ nohup: 忽略输入并把输出追加到"nohup.out"
[opensource@bigdata02 bin]$ ls
install-plugin.sh seatunnel-cluster.sh start-seatunnel-flink-13-connector-v2.sh start-seatunnel-spark-2-connector-v2.sh stop-seatunnel-cluster.sh
nohup.out seatunnel.sh start-seatunnel-flink-15-connector-v2.sh start-seatunnel-spark-3-connector-v2.sh
[opensource@bigdata02 bin]$ tail -f nohup.out
十二月 08, 2023 9:30:58 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Using configuration file at /home/opensource/app/apache-seatunnel-2.3.3/config/hazelcast.yaml
十二月 08, 2023 9:30:59 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Loading configuration '/home/opensource/app/apache-seatunnel-2.3.3/config/seatunnel.yaml' from System property 'seatunnel.config'
十二月 08, 2023 9:30:59 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Using configuration file at /home/opensource/app/apache-seatunnel-2.3.3/config/seatunnel.yaml
十二月 08, 2023 9:30:59 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Loading configuration '/home/opensource/app/apache-seatunnel-2.3.3/config/hazelcast.yaml' from System property 'hazelcast.config'
十二月 08, 2023 9:30:59 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Using configuration file at /home/opensource/app/apache-seatunnel-2.3.3/config/hazelcast.yaml
^C
[opensource@bigdata02 bin]$ tail -f ../logs/seatunnel-engine-server.log
Member [localhost]:5801 - 9db58520-2335-441c-8999-ad78123f6509 this
]
2023-12-08 09:30:59,930 INFO com.hazelcast.core.LifecycleService - [localhost]:5801 [seatunnel] [5.1] [localhost]:5801 is STARTED
2023-12-08 09:31:01,865 INFO org.apache.seatunnel.engine.server.CoordinatorService - [localhost]:5801 [seatunnel] [5.1] This node become a new active master node, begin init coordinator service
2023-12-08 09:31:02,061 INFO com.hazelcast.internal.partition.impl.PartitionStateManager - [localhost]:5801 [seatunnel] [5.1] Initializing cluster partition table arrangement...
2023-12-08 09:31:04,863 INFO org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - Init ResourceManager
2023-12-08 09:31:04,863 INFO org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - initWorker...
2023-12-08 09:31:04,864 INFO org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - initWorker live nodes: [[localhost]:5801]
2023-12-08 09:31:04,865 INFO org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - registerWorker: {[localhost]:5801=WorkerProfile(address=[localhost]:5801, profile=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=2058354688}}, unassignedResource=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=2058354688}}, assignedSlots=[], unassignedSlots=[])}
2023-12-08 09:31:59,863 INFO org.apache.seatunnel.engine.server.CoordinatorService - [localhost]:5801 [seatunnel] [5.1]
***********************************************
CoordinatorService Thread Pool Status
***********************************************
activeCount : 0
corePoolSize : 0
maximumPoolSize : 2147483647
poolSize : 0
completedTaskCount : 0
taskCount : 0
***********************************************
2023-12-08 09:31:59,867 INFO org.apache.seatunnel.engine.server.CoordinatorService - [localhost]:5801 [seatunnel] [5.1]
***********************************************
Job info detail
***********************************************
createdJobCount : 0
scheduledJobCount : 0
runningJobCount : 0
failingJobCount : 0
failedJobCount : 0
cancellingJobCount : 0
canceledJobCount : 0
finishedJobCount : 0
restartingJobCount : 0
suspendedJobCount : 0
reconcilingJobCount : 0
***********************************************
手动安装
不想全部安装，修改配置：vi config/plugin_config
无法在线安装，只能手动安装，maven库下载地址：https://repo.maven.apache.org/maven2/org/apache/seatunnel/
对于自动下载下来的jar包，已经全部打包上传到CSDN,大家可以免费下载，直接放到connectors/seatunnel
下载地址：https://download.csdn.net/download/qq_36434219/88609069
CSDN资源大小不允许超过1000M,这里采用.tar.xz格式（解压会有些慢，小十分钟吧）
三种压缩命令：
<code>tar -cf seatunnel_connectors_2.3.3.tar seatunnel
tar -czf seatunnel_connectors_2.3.3.tar.gz seatunnel
tar -cJf seatunnel_connectors_2.3.3.tar.xz seatunnel
对应解压命令：
tar -xf seatunnel_connectors_2.3.3.tar
tar -xzf seatunnel_connectors_2.3.3.tar.gz
tar -xJf seatunnel_connectors_2.3.3.tar.xz
Web界面部署
上传解压web部署包：<code>tar xf apache-seatunnel-web-1.0.0-bin.tar.gz
初始化数据库
初始化SQL脚本 script/seatunnel_server_mysql.sql
<code>SET FOREIGN_KEY_CHECKS = 0;
-- ----------------------------
-- Table structure for role
-- ----------------------------
DROP TABLE IF EXISTS `role`;
CREATE TABLE `role` (
`id` int(20) NOT NULL AUTO_INCREMENT,
`type` int(2) NOT NULL,
`role_name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL,
`description` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NULL DEFAULT NULL,
`create_time` datetime(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
`update_time` datetime(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB AUTO_INCREMENT = 1 CHARACTER SET = utf8mb4 COLLATE = utf8mb4_general_ci ROW_FORMAT = Dynamic;
-- ----------------------------
-- Records of role
-- ----------------------------
INSERT INTO `role`(`type`,`role_name`,`description`) values (0, 'ADMIN_ROLE', 'Admin User');
INSERT INTO `role`(`type`,`role_name`,`description`) values (1, 'NORMAL_ROLE', 'Normal User');
-- ----------------------------
-- Table structure for role_user_relation
-- ----------------------------
DROP TABLE IF EXISTS `role_user_relation`;
CREATE TABLE `role_user_relation` (
`id` int(20) NOT NULL AUTO_INCREMENT,
`role_id` int(20) NOT NULL,
`user_id` int(20) NOT NULL,
`create_time` datetime(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
`update_time` datetime(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB AUTO_INCREMENT = 1 CHARACTER SET = utf8mb4 COLLATE = utf8mb4_general_ci ROW_FORMAT = Dynamic;
-- ----------------------------
-- Table structure for t_st_datasource
-- ----------------------------
DROP TABLE IF EXISTS `t_st_datasource`;
CREATE TABLE `t_st_datasource` (
`id` bigint(20) NOT NULL,
`datasource_name` varchar(63) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
`plugin_name` varchar(63) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
`plugin_version` varchar(63) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT '1.0.0',
`datasource_config` varchar(1023) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
`description` varchar(63) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
`create_user_id` int(11) NOT NULL,
`update_user_id` int(11) NOT NULL,
`create_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
`update_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
PRIMARY KEY (`id`) USING BTREE,
UNIQUE INDEX `t_st_datasource_datasource_name_uindex`(`datasource_name`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Dynamic;
-- ----------------------------
-- Table structure for t_st_job_definition
-- ----------------------------
DROP TABLE IF EXISTS `t_st_job_definition`;
CREATE TABLE `t_st_job_definition` (
`id` bigint(20) NOT NULL,
`name` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
`description` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
`job_type` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
`create_user_id` int(11) NOT NULL,
`update_user_id` int(11) NOT NULL,
`create_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
`update_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
PRIMARY KEY (`id`) USING BTREE,
UNIQUE INDEX `name`(`name`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Dynamic;
-- ----------------------------
-- Table structure for t_st_job_instance
-- ----------------------------
DROP TABLE IF EXISTS `t_st_job_instance`;
CREATE TABLE `t_st_job_instance` (
`id` bigint(20) NOT NULL,
`job_define_id` bigint(20) NOT NULL,
`job_status` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
`job_config` text CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
`engine_name` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
`engine_version` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
`job_engine_id` varchar(200) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
`create_user_id` int(20) NOT NULL,
`update_user_id` int(20) NULL DEFAULT NULL,
`create_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
`update_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
`end_time` timestamp(3) NULL DEFAULT NULL,
`job_type` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Dynamic;
-- ----------------------------
-- Table structure for t_st_job_instance_history
-- ----------------------------
DROP TABLE IF EXISTS `t_st_job_instance_history`;
CREATE TABLE `t_st_job_instance_history` (
`id` bigint(20) NOT NULL,
`dag` text CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
`create_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
`update_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Dynamic;
-- ----------------------------
-- Table structure for t_st_job_line
-- ----------------------------
DROP TABLE IF EXISTS `t_st_job_line`;
CREATE TABLE `t_st_job_line` (
`id` bigint(20) NOT NULL,
`version_id` bigint(20) NOT NULL,
`input_plugin_id` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
`target_plugin_id` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
`create_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
`update_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
PRIMARY KEY (`id`) USING BTREE,
INDEX `job_line_version_index`(`version_id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Dynamic;
-- ----------------------------
-- Table structure for t_st_job_metrics
-- ----------------------------
DROP TABLE IF EXISTS `t_st_job_metrics`;
CREATE TABLE `t_st_job_metrics` (
`id` bigint(20) NOT NULL,
`job_instance_id` bigint(20) NOT NULL,
`pipeline_id` int(20) NOT NULL,
`read_row_count` bigint(20) NOT NULL,
`write_row_count` bigint(20) NOT NULL,
`source_table_names` varchar(200) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
`sink_table_names` varchar(200) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
`read_qps` bigint(20) NULL DEFAULT NULL,
`write_qps` bigint(20) NULL DEFAULT NULL,
`record_delay` bigint(20) NULL DEFAULT NULL,
`status` varchar(20) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
`create_user_id` int(20) NOT NULL,
`update_user_id` int(20) NULL DEFAULT NULL,
`create_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
`update_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Dynamic;
-- ----------------------------
-- Table structure for t_st_job_task
-- ----------------------------
DROP TABLE IF EXISTS `t_st_job_task`;
CREATE TABLE `t_st_job_task` (
`id` bigint(20) NOT NULL,
`version_id` bigint(20) NOT NULL,
`plugin_id` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
`name` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
`config` text CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL,
`transform_options` varchar(5000) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
`output_schema` text CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL,
`connector_type` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
`datasource_id` bigint(20) NULL DEFAULT NULL,
`datasource_option` varchar(5000) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
`select_table_fields` varchar(5000) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
`scene_mode` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
`type` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
`create_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
`update_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
PRIMARY KEY (`id`) USING BTREE,
INDEX `job_task_plugin_id_index`(`plugin_id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Dynamic;
-- ----------------------------
-- Table structure for t_st_job_version
-- ----------------------------
DROP TABLE IF EXISTS `t_st_job_version`;
CREATE TABLE `t_st_job_version` (
`id` bigint(20) NOT NULL,
`job_id` bigint(20) NOT NULL,
`name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
`job_mode` varchar(10) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
`env` text CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL,
`engine_name` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
`engine_version` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
`create_user_id` int(11) NOT NULL,
`update_user_id` int(11) NOT NULL,
`create_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
`update_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Dynamic;
-- ----------------------------
-- Table structure for t_st_virtual_table
-- ----------------------------
DROP TABLE IF EXISTS `t_st_virtual_table`;
CREATE TABLE `t_st_virtual_table` (
`id` bigint(20) NOT NULL,
`datasource_id` bigint(20) NOT NULL,
`virtual_database_name` varchar(63) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
`virtual_table_name` varchar(63) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
`table_fields` varchar(1023) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
`virtual_table_config` varchar(1023) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
`description` varchar(63) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
`create_user_id` int(11) NOT NULL,
`update_user_id` int(11) NOT NULL,
`create_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
`update_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Dynamic;
-- ----------------------------
-- Table structure for user
-- ----------------------------
DROP TABLE IF EXISTS `user`;
CREATE TABLE `user` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`username` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL,
`password` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL,
`status` tinyint(4) NOT NULL,
`type` tinyint(4) NOT NULL,
`create_time` datetime(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
`update_time` datetime(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB AUTO_INCREMENT = 2 CHARACTER SET = utf8mb4 COLLATE = utf8mb4_general_ci ROW_FORMAT = Dynamic;
-- ----------------------------
-- Table structure for user_login_log
-- ----------------------------
DROP TABLE IF EXISTS `user_login_log`;
CREATE TABLE `user_login_log` (
`id` bigint(20) NOT NULL AUTO_INCREMENT,
`user_id` int(11) NOT NULL,
`token` mediumtext CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL,
`token_status` tinyint(1) NOT NULL,
`create_time` datetime(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
`update_time` datetime(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB AUTO_INCREMENT = 106 CHARACTER SET = utf8mb4 COLLATE = utf8mb4_general_ci ROW_FORMAT = Dynamic;
-- ----------------------------
-- Records of user_login_log
-- ----------------------------
INSERT INTO `user`(`username`,`password`,`status`,`type`) values ('admin', '7f97da8846fed829bb8d1fd9f8030f3b', 0, 0);
SET FOREIGN_KEY_CHECKS = 1;
配置文件修改
配置数据库信息： vi conf/application.yml
复制引擎服务中配置文件到Web配置目录下面
<code>[opensource@bigdata02 app]$ cp apache-seatunnel-2.3.3/config/hazelcast-client.yaml apache-seatunnel-web-1.0.0-bin/conf/
[opensource@bigdata02 app]$ cp apache-seatunnel-2.3.3/connectors/plugin-mapping.properties apache-seatunnel-web-1.0.0-bin/conf/
配置下载数据源jar包
选择的Mysql作为元数据库，对应的驱动包<code>mysql-connector-java-8.0.25.jar放到libs下
下载其它数据源资源包,首先下载源码，然后执行下载脚本<code>download_datasource.sh(本地安装过git)
源码地址:https://github.com/apache/seatunnel-web
修改<code>download_datasource.sh相关配置
执行<code>download_datasource.sh
开始下载jar包
下载完成
上传到libs目录
启动服务
<code>sh bin/seatunnel-backend-daemon.sh start
Web页面一览
登录
登录地址：http://xxxx:8801/ui/#/login ;
用户密码 admin/admin
数据源
任务
尝试创建整库同步
用户管理
虚拟表
其他
问题记录
服务启动失败：Application run failed org.yaml.snakeyaml.scanner.ScannerException: while scanning an anchor
<code>14:29:08.032 [main] ERROR org.springframework.boot.SpringApplication - Application run failed
org.yaml.snakeyaml.scanner.ScannerException: while scanning an anchor
in 'reader', line 30, column 15:
password: &R7#J*vDVWEuT48m
^
unexpected character found *(42)
in 'reader', line 30, column 20:
password: &R7#J*vDVWEuT48m
^
at org.yaml.snakeyaml.scanner.ScannerImpl.scanAnchor(ScannerImpl.java:1507)
at org.yaml.snakeyaml.scanner.ScannerImpl.fetchAnchor(ScannerImpl.java:958)
at org.yaml.snakeyaml.scanner.ScannerImpl.fetchMoreTokens(ScannerImpl.java:389)
at org.yaml.snakeyaml.scanner.ScannerImpl.checkToken(ScannerImpl.java:248)
at org.yaml.snakeyaml.parser.ParserImpl$ParseBlockMappingValue.produce(ParserImpl.java:665)
at org.yaml.snakeyaml.parser.ParserImpl.peekEvent(ParserImpl.java:165)
at org.yaml.snakeyaml.comments.CommentEventsCollector$1.peek(CommentEventsCollector.java:59)
at org.yaml.snakeyaml.comments.CommentEventsCollector$1.peek(CommentEventsCollector.java:45)
at org.yaml.snakeyaml.comments.CommentEventsCollector.collectEvents(CommentEventsCollector.java:140)
at org.yaml.snakeyaml.comments.CommentEventsCollector.collectEvents(CommentEventsCollector.java:119)
at org.yaml.snakeyaml.composer.Composer.composeScalarN
解决办法： 密码&R7#J*vDVWEuT48m增加双引号
原文链接：https://blog.csdn.net/qq_36434219/article/details/134857560
本文由 白鲸开源科技 提供发布支持！
上一篇： 【springboot】Failed to start bean ‘webServerStartStop‘；
下一篇： 【UE5】【插件】【WebUI】内置浏览器插件WebUI简易使用笔记
本文标签
声明
本文内容仅代表作者观点，或转载于其他网站，本站不以此文作为商业用途
如有涉及侵权，请联系本站进行删除
转载本站原创文章，请注明来源及作者。