SeaTunnel Web安装 一把成
崔小磊 2024-07-16 13:33:04 阅读 90
安装相关jar包，以及SeaTunnel 和Web 打成的包，可以直接使用，但是需要安装MySQL客户端的分享：
链接：https://pan.baidu.com/s/1qrt1RAX38SgIpNklbQJ7pA
提取码：0kmf
1. 环境准备
| 环境名称 | 版本 |
| 系统环境 | CentOS7.9 |
| Java环境 | JDK1.8 |
1.1 系统环境
略
1.2 Java环境安装
下载jdk1.8安装包并拷贝到服务器
<code># 进入文件夹
cd /usr/local
# 将jdk拷贝到该位置
# 解压文件
tar -zxvf jdk-8u381-linux-x64.tar.gz
修改环境变量
vim /etc/profile
在环境变量中添加Java环境
export JAVA_HOME=/usr/local/jdk1.8.0_381/
export CLASSPATH=$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export PATH=$PATH:$JAVA_HOME/bin
重新加载环境变量
source /etc/profile
验证Java环境
java -version
1.3 SeaTunnel单节点安装
下载安装包
下载地址：
Apache SeaTunnel
下载SeaTunnel安装包
下载SeaTunnel Web安装包
创建安装SeaTunnel 目录
<code># 创建用于安装SeaTunnel 后台引擎的目录
mkdir -p /opt/seatunnel/backend
# 创建用户安装SeaTunnel Web前台页面的目录
mkdir -p /opt/seatunnel/web
拷贝并安装SeaTunnel
# 进入到SeaTunnel 后台安装目录，将下载好的安装包拷贝到该目录
[root@localhost backend]# pwd
/opt/seatunnel/backend
[root@localhost backend]# ls
apache-seatunnel-2.3.3-bin.tar.gz
# 安装SeaTunnel 后台
tar -zxvf apache-seatunnel-2.3.3-bin.tar.gz
# 进入到SeaTunnel Web页面程序安装目录，将下载号的安装包拷贝到该目录
[root@localhost web]# pwd
/opt/seatunnel/web
[root@localhost web]# ls
apache-seatunnel-web-1.0.0-bin.tar.gz
# 安装SeaTunnel Web页面程序
tar -zxvf apache-seatunnel-web-1.0.0-bin.tar.gz
将SeaTunnel 后台配置到环境变量中
# 打开环境变量文件
vim /etc/profile
# 在环境变量文件最后添加如下配置
export SEATUNNEL_HOME=/opt/seatunnel/backend/apache-seatunnel-2.3.3
export PATH=$PATH:$SEATUNNEL_HOME/bin
# 重新加载环境变量文件
source /etc/profile
安装需要的连接器
# 进入到seatunnel 安装目录下
[root@localhost seatunnel]# cd apache-seatunnel-2.3.3
# 进入到插件配置文件中
[root@localhost apache-seatunnel-2.3.3]# vim config/plugin_config
# 删除多余连接器，留下需要用到的连接器
--connectors-v2--
connector-cdc-mysql
connector-cdc-sqlserver
connector-doris
connector-elasticsearch
connector-jdbc
--end--
# 退出保存连接器配置文件
:qw
# 在线安装拦截器
[root@localhost apache-seatunnel-2.3.3]# sh bin/install-plugin.sh
# 等待安装，安装完成后可查看已经安装的连接器
[root@localhost apache-seatunnel-2.3.3]# ls connectors/seatunnel/*
connectors/seatunnel/connector-cdc-mysql-2.3.3.jar
connectors/seatunnel/connector-cdc-sqlserver-2.3.3.jar
connectors/seatunnel/connector-console-2.3.3.jar
connectors/seatunnel/connector-doris-2.3.3.jar
connectors/seatunnel/connector-elasticsearch-2.3.3.jar
connectors/seatunnel/connector-fake-2.3.3.jar
connectors/seatunnel/connector-jdbc-2.3.3.jar
注意：
connector-jdbc 连接器可以连接多种使用JDBC进行连接的多种数据库，如下：
| MySQL、PostgreSQL、DM、Phoenix 、SQLServer、Oracle、sqlite、GBase8a、StarRocks、db2、saphana、Doris、teradata、Redshift、Snowflake、Vertica、OceanBase
|
拷贝需要的数据库连接jar包
cd /opt/seatunnel/backend/apache-seatunnel-2.3.3
# 创建连接器对应的目录，用于存放数据库连接驱动
[root@localhost apache-seatunnel-2.3.3]# mkdir -p plugins/jdbc/lib/
# 将下载的数据库连接包上传到该目录下
[root@localhost lib]# pwd
/opt/seatunnel/backend/apache-seatunnel-2.3.3/plugins/jdbc/lib
[root@localhost lib]# ls
mysql-connector-java-8.0.28.jar ojdbc8.jar postgresql-42.6.0.jar
数据库连接包下载地址：
| 数据库
| 下载地址
|
| mysql
| https://mvnrepository.com/artifact/mysql/mysql-connector-java
|
| oracle
| https://mvnrepository.com/artifact/com.oracle.database.jdbc/ojdbc8
|
| postgresql
| https://mvnrepository.com/artifact/org.postgresql/postgresql
|
| doris
| https://mvnrepository.com/artifact/mysql/mysql-connector-java
|
测试验证
# 进入SeaTunnel 后台安装目录
cd /opt/seatunnel/backend/apache-seatunnel-2.3.3
# 跑样例脚本
sh bin/seatunnel.sh --config config/v2.batch.config.template -e local
出现以下信息表示验证成功
后台启用服务
cd /opt/seatunnel/backend/apache-seatunnel-2.3.3
# 启动服务
nohup sh bin/seatunnel-cluster.sh 2>&1 &
查看SeaTunnel 运行日志
# 查看SeaTunnel运行日志
tail -100f logs/seatunnel-engine-server.log
日志中有以下信息表示后台启动成功
1.4 安装SeaTunnel Web
注意：必须保证SeaTunnel的后台引擎服务正常运行，Web端才可以正常运行。
安装MySQL客户端
因为SeaTunnel Web 需要连接到MySQL数据库，并初始化数据库，所以需要安装MySQL Client。
<code># 先检查服务器中是否存在mariadb
rpm -qa|grep mariadb
#删除相关依赖包（若不存在，可以跳过）
rpm -e --nodeps mariadb-libs
#检查是否删除干净
rpm -qa|grep mariadb
# 检查MySQL
rpm -qa|grep mysql
# 创建一个目录用于存放MySQL Client 安装文件
mkdir -p /opt/mysqlrpm
# 将安装包拷贝到该目录下
[root@localhost mysqlrpm]# pwd
/opt/mysqlrpm
[root@localhost mysqlrpm]# ls
lrzsz-0.12.20-36.el7.x86_64.rpm
mysql-community-client-5.7.40-1.el7.x86_64.rpm
mysql-community-common-5.7.40-1.el7.x86_64.rpm
mysql-community-libs-5.7.40-1.el7.x86_64.rpm
# 安装lrzsz
rpm -ivh lrzsz-0.12.20-36.el7.x86_64.rpm
# 按照顺序安装MySQL Client
rpm -ivh mysql-community-common-5.7.40-1.el7.x86_64.rpm
rpm -ivh mysql-community-libs-5.7.40-1.el7.x86_64.rpm
rpm -ivh mysql-community-client-5.7.40-1.el7.x86_64.rpm
修改SeaTunnel Web 下script/seatunnel_server_env.sh中连接数据库信息
# 进入到SeaTunnel Web 安装目录
cd /opt/seatunnel/web/apache-seatunnel-web-1.0.0-bin
# 备份script/seatunnel_server_env.sh文件
cp script/seatunnel_server_env.sh script/seatunnel_server_env.sh.bak
# 修改script/seatunnel_server_env.sh文件
vim script/seatunnel_server_env.sh
# 修改以下配置信息
export HOSTNAME="127.0.0.1"code>
export PORT="3306"code>
export USERNAME="root"code>
export PASSWORD="123456"code>
# 修改如下,加上了前缀，以防与系统中其它配置冲突
# 将数据库连接信息修改成你要链接的信息
export STWEB_HOSTNAME="127.0.0.1"code>
export STWEB_PORT="3306"code>
export STWEB_USERNAME="root"code>
export STWEB_PASSWORD="123456"code>
修改SeaTunnel Web 下script/init_sql.sh 中连接数据库信息
vim script/init_sql.sh
主要是保证里边的${XXXXX}的配置信息与script/seatunnel_server_env.sh中的保持一致。
执行数据库初始化脚本
cd /opt/seatunnel/web/apache-seatunnel-web-1.0.0-bin
# 执行初始化文件
sh script/init_sql.sh
修改SeaTunnel Web 连接MySQL数据库的配置信息
# 进入SeaTunnel Web 安装目录
cd /opt/seatunnel/web/apache-seatunnel-web-1.0.0-bin
# 打开SeaTunnel Web 配置文件
vim conf/application.yml
# 修改以下部分配置：端口号，MySQL连接IP，MySQL用户名和密码
server:
#port: 8801
port: 8882
spring:
application:
name: seatunnel
jackson:
date-format: yyyy-MM-dd HH:mm:ss
datasource:
driver-class-name: com.mysql.jdbc.Driver
url: jdbc:mysql://192.168.1.111:3306/seatunnel?useSSL=false&useUnicode=true&characterEnc
oding=utf-8&allowMultiQueries=true&allowPublicKeyRetrieval=true
username: root
password: 123456
配置client信息
# 将seatunnel后台的两个文件拷贝到web的conf目录下
cp /opt/seatunnel/backend/apache-seatunnel-2.3.3/config/hazelcast-client.yaml conf/
cp /opt/seatunnel/backend/apache-seatunnel-2.3.3/connectors/plugin-mapping.properties conf/
下载Web程序需要的数据源（数据源需要的各种jar包）
这里在各种jar包，如果缺失的话，在Web页面创建数据源时会出现空数据源的情况。
下载jar包
首先你自己的电脑上安装了github
下载jar包额度脚本
我是在我电脑的D盘根目录下，创建了download_datasource.sh文件，文件内容如下：
有两处需要修改，请看注释。
<code>#!/bin/bash
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
#This script is used to download the connector plug-ins required during the running process.
#All are downloaded by default. You can also choose what you need.
#You only need to configure the plug-in name in config/plugin_config.
# get seatunnel web home
# SEATUNNEL_WEB_HOME=$(cd $(dirname $0);cd ../;pwd)
# 此处需要修改成你本地的目录
DATASOURCE_DIR=D:/down_jars
# If you don’t want to download a certain data source, you can delete the element below
datasource_list=(
"datasource-plugins-api"
"datasource-elasticsearch"
"datasource-hive"
"datasource-jdbc-clickhouse"
"datasource-jdbc-hive"
"datasource-jdbc-mysql"
"datasource-jdbc-oracle"
"datasource-jdbc-postgresql"
"datasource-jdbc-redshift"
"datasource-jdbc-sqlserver"
"datasource-jdbc-starrocks"
"datasource-jdbc-tidb"
"datasource-kafka"
"datasource-mysql-cdc"
"datasource-s3"
"datasource-sqlserver-cdc"
"datasource-starrocks"
)
# the datasource default version is 1.0.0, you can also choose a custom version. eg: 1.1.2: sh install-datasource.sh 2.1.2
version=1.0.0
if [ -n "$1" ]; then
version="$1"code>
fi
echo "Downloading SeaTunnel Web Datasource lib, usage version is ${version}"
# create the datasource directory
if [ ! -d "$DATASOURCE_DIR" ];
then
mkdir -p "$DATASOURCE_DIR"
echo "Created datasource directory."
fi
for i in "${datasource_list[@]}"
do
echo "$i"
echo "Downloading datasource: " "$i"
# 修改成mvn
mvn dependency:get -DgroupId=org.apache.seatunnel -DartifactId="$i" -Dversion="$version" -Ddest="$DATASOURCE_DIR"code>
done
之后在download_datasource.sh目录下点击鼠标右键，选择Git Bash Here
之后在打开的页面执行 sh download_datasource.sh 点击回车，等待下载完成。
之后将下载下来的包，拷贝到SeaTunnel Web 程序的 /libs目录下。
将MySQL Jar 包拷贝到libs目录下
将mysql连接数据库的Jar 拷贝到 /libs 目录下，否则Web程序启动时会报错，连不上数据库。
启动SeaTunnel Web 程序
cd /opt/seatunnel/web/apache-seatunnel-web-1.0.0-bin
# 启动SeaTunnel Web
sh bin/seatunnel-backend-daemon.sh start
# 查看启动日志
tail -100f logs/seatunnel-web.log
配置开放端口号(两种方式)
#-------------iptables-----------------
#开打端口号开放配置文件
vim /etc/sysconfig/iptables
# 添加一行
-A INPUT -p tcp -m state --state NEW -m tcp --dport 8882 -j ACCEPT
# 重启启动防火墙
systemctl restart iptables.service
#------------firewall-------------------
# 依次执行下面命令
firewall-cmd --zone=public --add-port=8882/tcp --permanent
firewall-cmd --reload
firewall-cmd --list-all
查看日志文件
访问SeaTunnel Web
http://192.168.182.166:8882
默认用户名密码：admin/admin
到此为止SeaTunnel 与SeaTunnel Web 已经安装完成。
声明
本文内容仅代表作者观点，或转载于其他网站，本站不以此文作为商业用途
如有涉及侵权，请联系本站进行删除
转载本站原创文章，请注明来源及作者。