基于 amd64 编译 Hadoop 3.3.6 Native Libraries 完整套件

官方的预构建包对很多 Native Libraries 功能扩展支持不是很完善，需要重新进行构建，本文演示 RHEL 8 基于 amd64 架构环境编译。

过程

基础环境 RHEL 8 (amd64) 最小化安装实例。激活和订阅步骤在此省略，编译前完整升级一遍系统。

sudo dnf update -y

升级完毕后检查系统版本（如果提示命令未找到，手动安装 redhat-lsb-core）

$ lsb_release -a
LSB Version:    :core-4.1-amd64:core-4.1-noarch
Distributor ID: RedHatEnterprise
Description:    Red Hat Enterprise Linux release 8.8 (Ootpa)
Release:        8.8
Codename:       Ootpa

当前版本 RHEL 8.8 (Ootpa) 重启后继续操作。

环境

JDK

因为 Hadoop 及相关套件都是基于 Java 编写的，先安装基础环境。理论上 OpenJDK 亦可使用，不过谨慎起见，在 Oracle JDK 官网下载 JDK 安装包，或者也可以使用 Liberica JDK 版本。

## 若下载版本为 8u361 （不同架构仅文件后缀略有不同）
sudo dnf localinstall jdk-8u361-linux-x64.rpm

安装后检查版本

$ java -version
java version "1.8.0_361"
Java(TM) SE Runtime Environment (build 1.8.0_361-b09)
Java HotSpot(TM) 64-Bit Server VM (build 25.361-b09, mixed mode)

Maven

然后部署 Maven ，提供 Java 构建环境。

wget https://dlcdn.apache.org/maven/maven-3/3.9.4/binaries/apache-maven-3.9.4-bin.tar.gz
sudo tar xf apache-maven-3.9.4-bin.tar.gz -C /opt/

添加全局环境变量

sudo vim /etc/profile.d/maven.sh
## 添加以下内容
export JAVA_HOME="/usr/java/jdk1.8.0_361-amd64"
export PATH=$JAVA_HOME/bin:$PATH
export M2_HOME="/opt/apache-maven-3.9.4"
export MAVEN_HOME="/opt/apache-maven-3.9.4"
export PATH=$M2_HOME/bin:$PATH
## 保存后生效环境环境变量或者重新登录终端
source /etc/profile

System Depends

然后准备 Native Libraries 编译环境，官方演示编译环境为 Ubuntu，因此依赖需要更换为红帽系的包名。

sudo dnf groupinstall 'Development Tools' -y

在 RHEL 中，许多开发库是无法直接安装的，因为默认的仓库不含，需要启用额外的仓库 CRB (CodeReady Linux Builder)，使用官方订阅工具添加 CRB 仓库：

sudo subscription-manager repos --enable codeready-builder-for-rhel-8-x86_64-rpms

然后安装 Native Depends

sudo dnf install -y bzip2-devel \
                    cyrus-sasl-devel \
                    fuse-devel \
                    libzstd-devel \
                    libtirpc-devel \
                    openssl-devel \
                    protobuf-devel \
                    snappy-devel \
                    zlib-devel

Build Tools

接下来安装编译工具

sudo dnf install -y cmake expect

安装完成后检查系统组件版本

$ cmake --version
cmake version 3.20.2

CMake suite maintained and supported by Kitware (kitware.com/cmake).
$ protoc --version
libprotoc 3.5.0

构建

下载 Hadoop 3.3.6 版本源码包，解压并开始构建

wget https://dlcdn.apache.org/hadoop/common/hadoop-3.3.6/hadoop-3.3.6-src.tar.gz
tar xf hadoop-3.3.6-src.tar.gz
cd hadoop-3.3.6-src/
mvn clean package -Pdist,native, -DskipTests -Dtar -Dmaven.javadoc-skip=true -X

看到以下提示即为构建成功。

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Apache Hadoop Main 3.3.6:
[INFO] 
[INFO] Apache Hadoop Main ................................. SUCCESS [  1.309 s]
[INFO] Apache Hadoop Build Tools .......................... SUCCESS [  1.505 s]
[INFO] Apache Hadoop Project POM .......................... SUCCESS [  1.092 s]
[INFO] Apache Hadoop Annotations .......................... SUCCESS [  1.590 s]
[INFO] Apache Hadoop Assemblies ........................... SUCCESS [  0.108 s]
[INFO] Apache Hadoop Project Dist POM ..................... SUCCESS [  0.986 s]
[INFO] Apache Hadoop Maven Plugins ........................ SUCCESS [  2.209 s]
[INFO] Apache Hadoop MiniKDC .............................. SUCCESS [  0.799 s]
[INFO] Apache Hadoop Auth ................................. SUCCESS [  4.329 s]
[INFO] Apache Hadoop Auth Examples ........................ SUCCESS [  1.519 s]
[INFO] Apache Hadoop Common ............................... SUCCESS [ 21.880 s]
[INFO] Apache Hadoop NFS .................................. SUCCESS [  3.661 s]
[INFO] Apache Hadoop KMS .................................. SUCCESS [  3.173 s]
[INFO] Apache Hadoop Registry ............................. SUCCESS [  3.491 s]
[INFO] Apache Hadoop Common Project ....................... SUCCESS [  0.030 s]
[INFO] Apache Hadoop HDFS Client .......................... SUCCESS [ 15.645 s]
[INFO] Apache Hadoop HDFS ................................. SUCCESS [ 41.532 s]
[INFO] Apache Hadoop HDFS Native Client ................... SUCCESS [01:49 min]
[INFO] Apache Hadoop HttpFS ............................... SUCCESS [  4.299 s]
[INFO] Apache Hadoop HDFS-NFS ............................. SUCCESS [  3.031 s]
[INFO] Apache Hadoop HDFS-RBF ............................. SUCCESS [ 19.019 s]
[INFO] Apache Hadoop HDFS Project ......................... SUCCESS [  0.047 s]
[INFO] Apache Hadoop YARN ................................. SUCCESS [  0.028 s]
[INFO] Apache Hadoop YARN API ............................. SUCCESS [  8.755 s]
[INFO] Apache Hadoop YARN Common .......................... SUCCESS [ 16.161 s]
[INFO] Apache Hadoop YARN Server .......................... SUCCESS [  0.030 s]
[INFO] Apache Hadoop YARN Server Common ................... SUCCESS [  6.323 s]
[INFO] Apache Hadoop YARN NodeManager ..................... SUCCESS [ 26.985 s]
[INFO] Apache Hadoop YARN Web Proxy ....................... SUCCESS [  2.916 s]
[INFO] Apache Hadoop YARN ApplicationHistoryService ....... SUCCESS [  3.656 s]
[INFO] Apache Hadoop YARN Timeline Service ................ SUCCESS [  3.456 s]
[INFO] Apache Hadoop YARN ResourceManager ................. SUCCESS [ 12.786 s]
[INFO] Apache Hadoop YARN Server Tests .................... SUCCESS [  2.162 s]
[INFO] Apache Hadoop YARN Client .......................... SUCCESS [  4.194 s]
[INFO] Apache Hadoop YARN SharedCacheManager .............. SUCCESS [  2.842 s]
[INFO] Apache Hadoop YARN Timeline Plugin Storage ......... SUCCESS [  2.943 s]
[INFO] Apache Hadoop YARN TimelineService HBase Backend ... SUCCESS [  0.035 s]
[INFO] Apache Hadoop YARN TimelineService HBase Common .... SUCCESS [  3.157 s]
[INFO] Apache Hadoop YARN TimelineService HBase Client .... SUCCESS [  4.079 s]
[INFO] Apache Hadoop YARN TimelineService HBase Servers ... SUCCESS [  0.028 s]
[INFO] Apache Hadoop YARN TimelineService HBase Server 1.2  SUCCESS [  3.782 s]
[INFO] Apache Hadoop YARN TimelineService HBase tests ..... SUCCESS [  2.741 s]
[INFO] Apache Hadoop YARN Router .......................... SUCCESS [  3.466 s]
[INFO] Apache Hadoop YARN TimelineService DocumentStore ... SUCCESS [  2.705 s]
[INFO] Apache Hadoop YARN Applications .................... SUCCESS [  0.025 s]
[INFO] Apache Hadoop YARN DistributedShell ................ SUCCESS [  3.139 s]
[INFO] Apache Hadoop YARN Unmanaged Am Launcher ........... SUCCESS [  2.325 s]
[INFO] Apache Hadoop MapReduce Client ..................... SUCCESS [  1.501 s]
[INFO] Apache Hadoop MapReduce Core ....................... SUCCESS [  3.984 s]
[INFO] Apache Hadoop MapReduce Common ..................... SUCCESS [  4.637 s]
[INFO] Apache Hadoop MapReduce Shuffle .................... SUCCESS [  3.077 s]
[INFO] Apache Hadoop MapReduce App ........................ SUCCESS [  5.736 s]
[INFO] Apache Hadoop MapReduce HistoryServer .............. SUCCESS [  4.065 s]
[INFO] Apache Hadoop MapReduce JobClient .................. SUCCESS [  4.861 s]
[INFO] Apache Hadoop Mini-Cluster ......................... SUCCESS [  2.480 s]
[INFO] Apache Hadoop YARN Services ........................ SUCCESS [  0.026 s]
[INFO] Apache Hadoop YARN Services Core ................... SUCCESS [  3.236 s]
[INFO] Apache Hadoop YARN Services API .................... SUCCESS [  2.708 s]
[INFO] Apache Hadoop YARN Application Catalog ............. SUCCESS [  0.028 s]
[INFO] Apache Hadoop YARN Application Catalog Webapp ...... SUCCESS [ 20.952 s]
[INFO] Apache Hadoop YARN Application Catalog Docker Image  SUCCESS [  0.720 s]
[INFO] Apache Hadoop YARN Application MaWo ................ SUCCESS [  0.022 s]
[INFO] Apache Hadoop YARN Application MaWo Core ........... SUCCESS [  2.199 s]
[INFO] Apache Hadoop YARN Site ............................ SUCCESS [  0.025 s]
[INFO] Apache Hadoop YARN Registry ........................ SUCCESS [  1.135 s]
[INFO] Apache Hadoop YARN UI .............................. SUCCESS [  0.023 s]
[INFO] Apache Hadoop YARN CSI ............................. SUCCESS [  5.973 s]
[INFO] Apache Hadoop YARN Project ......................... SUCCESS [  9.065 s]
[INFO] Apache Hadoop MapReduce HistoryServer Plugins ...... SUCCESS [  2.356 s]
[INFO] Apache Hadoop MapReduce NativeTask ................. SUCCESS [ 27.164 s]
[INFO] Apache Hadoop MapReduce Uploader ................... SUCCESS [  2.329 s]
[INFO] Apache Hadoop MapReduce Examples ................... SUCCESS [  3.786 s]
[INFO] Apache Hadoop MapReduce ............................ SUCCESS [  4.007 s]
[INFO] Apache Hadoop MapReduce Streaming .................. SUCCESS [  4.321 s]
[INFO] Apache Hadoop Distributed Copy ..................... SUCCESS [  3.555 s]
[INFO] Apache Hadoop Client Aggregator .................... SUCCESS [  1.869 s]
[INFO] Apache Hadoop Dynamometer Workload Simulator ....... SUCCESS [  2.928 s]
[INFO] Apache Hadoop Dynamometer Cluster Simulator ........ SUCCESS [  3.328 s]
[INFO] Apache Hadoop Dynamometer Block Listing Generator .. SUCCESS [  2.716 s]
[INFO] Apache Hadoop Dynamometer Dist ..................... SUCCESS [  5.598 s]
[INFO] Apache Hadoop Dynamometer .......................... SUCCESS [  0.024 s]
[INFO] Apache Hadoop Archives ............................. SUCCESS [  2.771 s]
[INFO] Apache Hadoop Archive Logs ......................... SUCCESS [  3.062 s]
[INFO] Apache Hadoop Rumen ................................ SUCCESS [  3.874 s]
[INFO] Apache Hadoop Gridmix .............................. SUCCESS [  3.469 s]
[INFO] Apache Hadoop Data Join ............................ SUCCESS [  3.019 s]
[INFO] Apache Hadoop Extras ............................... SUCCESS [  3.201 s]
[INFO] Apache Hadoop Pipes ................................ SUCCESS [  2.583 s]
[INFO] Apache Hadoop Amazon Web Services support .......... SUCCESS [ 10.252 s]
[INFO] Apache Hadoop Kafka Library support ................ SUCCESS [  2.365 s]
[INFO] Apache Hadoop Azure support ........................ SUCCESS [  5.983 s]
[INFO] Apache Hadoop Aliyun OSS support ................... SUCCESS [  2.786 s]
[INFO] Apache Hadoop Scheduler Load Simulator ............. SUCCESS [  3.589 s]
[INFO] Apache Hadoop Resource Estimator Service ........... SUCCESS [  2.732 s]
[INFO] Apache Hadoop Azure Data Lake support .............. SUCCESS [  2.285 s]
[INFO] Apache Hadoop Image Generation Tool ................ SUCCESS [  3.409 s]
[INFO] Apache Hadoop Tools Dist ........................... SUCCESS [ 14.372 s]
[INFO] Apache Hadoop OpenStack support .................... SUCCESS [  0.043 s]
[INFO] Apache Hadoop Common Benchmark ..................... SUCCESS [  6.191 s]
[INFO] Apache Hadoop Tools ................................ SUCCESS [  0.023 s]
[INFO] Apache Hadoop Client API ........................... SUCCESS [01:05 min]
[INFO] Apache Hadoop Client Runtime ....................... SUCCESS [01:05 min]
[INFO] Apache Hadoop Client Packaging Invariants .......... SUCCESS [  0.592 s]
[INFO] Apache Hadoop Client Test Minicluster .............. SUCCESS [01:47 min]
[INFO] Apache Hadoop Client Packaging Invariants for Test . SUCCESS [  0.900 s]
[INFO] Apache Hadoop Client Packaging Integration Tests ... SUCCESS [  1.013 s]
[INFO] Apache Hadoop Distribution ......................... SUCCESS [ 30.781 s]
[INFO] Apache Hadoop Client Modules ....................... SUCCESS [  0.052 s]
[INFO] Apache Hadoop Tencent COS Support .................. SUCCESS [  3.627 s]
[INFO] Apache Hadoop Cloud Storage ........................ SUCCESS [  3.883 s]
[INFO] Apache Hadoop Cloud Storage Project ................ SUCCESS [  0.025 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  14:17 min
[INFO] Finished at: 2023-09-03T02:23:21+08:00
[INFO] ------------------------------------------------------------------------

生成的安装包在 hadoop-dist/target/ 目录下 hadoop-3.3.6.tar.gz 为最终编译的产品包。

可选项

ISA-L Support

ISA-L (Intelligent Storage Acceleration Library) 是 Intel 开发的智能存储加速库，可以为 HDFS 提高性能，可以在 ARMv8(aarch64) 和 AMD64(x86_64) 架构上编译。

此组件在编译时会自动检测并添加支持，因此请在编译前按下述步骤安装 ISA-L 库，然后进行编译即可原生支持 ISA-L。

## 安装依赖
sudo dnf install -y nasm
## 克隆源码
git clone https://github.com/intel/isa-l
## 编译
./autogen.sh
./configure --prefix=/usr --libdir=/usr/lib64
make
sudo make install

PMDK Support

PMDK(Persistent Memory Development Kit) 利用 PMDK 用户态编程库进行数据读写，减小用户态、内核态切换与文件系统开销，提高集群的读写性能。

PMDK 扩展与其他扩展不同，即便系统内检测到相关依赖库也不会默认编译支持，需要在编译时增加参数重新进行编译，增加支持，安装所需依赖。

amd64(x86_64) 可以使用发行版的包管理器进行安装：

## 只安装运行依赖 Runtime （部署机器上安装）
sudo dnf install -y libpmem librpmem libpmemblk libpmemlog libpmemobj libpmempool pmempool
## 只安装开发套件 Development （编译机器上安装）
sudo dnf install -y libpmem-devel librpmem-devel libpmemblk-devel libpmemlog-devel libpmemobj-devel libpmemobj++-devel libpmempool-devel

arm64(aarch64) 没有预构建包，需要手动编译：

## 安装编译依赖
sudo dnf install -y ndctl-devel daxctl-devel pandoc cmake gcc-c++
## 克隆源码
git clone https://github.com/pmem/pmdk
## 编译
make
sudo make install prefix=/usr

然后使用命令进行编译

mvn clean package -Pdist,native, -DskipTests -Dtar -Dmaven.javadoc-skip=true -Drequire.pmdk -X

编译后使用新安装包部署后重新执行检查

$ hadoop checknative
2023-09-16 15:16:00,763 INFO bzip2.Bzip2Factory: Successfully loaded & initialized native-bzip2 library system-native
2023-09-16 15:16:00,764 INFO zlib.ZlibFactory: Successfully loaded & initialized native-zlib library
2023-09-16 15:16:00,793 INFO nativeio.NativeIO: The native code was built with PMDK support, and PMDK libs were loaded successfully.
Native library checking:
hadoop:  true /opt/hadoop-3.3.6/lib/native/libhadoop.so.1.0.0
zlib:    true /lib64/libz.so.1
zstd  :  true /lib64/libzstd.so.1
bzip2:   true /lib64/libbz2.so.1
openssl: true /lib64/libcrypto.so
ISA-L:   true /lib64/libisal.so.2
PMDK:    true /usr/lib64/libpmem.so.1.0.0

常见问题

a) 编译报错 Failed to execute goal org.apache.hadoop:hadoop-maven-plugins:3.3.6:cmake-compile (cmake-compile) on project hadoop-pipes: CMake failed with error code 1 -> [Help 1]：

具体的报错类如：

[INFO] 
[INFO] --- hadoops:3.3.6:cmake-compile (cmake-compile) @ hadoop-pipes ---
[DEBUG] Loading mojo org.apache.hadoop:hadoop-maven-plugins:3.3.6:cmake-compile from plugin realm ClassRealm[plugin>org.apache.hadoop:hadoop-maven-plugins:3.3.6, parent: sun.misc.Launcher$AppClassLoader@7852e922]
[DEBUG] Configuring mojo execution 'org.apache.hadoop:hadoop-maven-plugins:3.3.6:cmake-compile:cmake-compile' with basic configurator -->
[DEBUG]   (f) output = /data/builder/hadoop-3.3.6-src/hadoop-tools/hadoop-pipes/target/native
[DEBUG]   (f) source = /data/builder/hadoop-3.3.6-src/hadoop-tools/hadoop-pipes/src
[DEBUG]   (f) vars = {JVM_ARCH_DATA_MODEL=64, OPENSSL_ROOT_DIR=null}
[DEBUG] -- end configuration --
[INFO] mkdirs '/data/builder/hadoop-3.3.6-src/hadoop-tools/hadoop-pipes/target/native'
[INFO] Running cmake /data/builder/hadoop-3.3.6-src/hadoop-tools/hadoop-pipes/src -DJVM_ARCH_DATA_MODEL=64 -G Unix Makefiles
[INFO] with extra environment variables {}
[WARNING] CMake Warning (dev) in CMakeLists.txt:
[WARNING]   No project() command is present.  The top-level CMakeLists.txt file must
[WARNING]   contain a literal, direct call to the project() command.  Add a line of
[WARNING]   code such as
[WARNING] 
[WARNING]     project(ProjectName)
[WARNING] 
[WARNING]   near the top of the file, but after cmake_minimum_required().
[WARNING] 
[WARNING]   CMake is pretending there is a "project(Project)" command on the first
[WARNING]   line.
[WARNING] This warning is for project developers.  Use -Wno-dev to suppress it.
[WARNING] 
[WARNING] -- The C compiler identification is GNU 8.5.0
[WARNING] -- The CXX compiler identification is GNU 8.5.0
[WARNING] -- Detecting C compiler ABI info
[WARNING] -- Detecting C compiler ABI info - done
[WARNING] -- Check for working C compiler: /usr/bin/cc - skipped
[WARNING] -- Detecting C compile features
[WARNING] -- Detecting C compile features - done
[WARNING] -- Detecting CXX compiler ABI info
[WARNING] -- Detecting CXX compiler ABI info - done
[WARNING] -- Check for working CXX compiler: /usr/bin/c++ - skipped
[WARNING] -- Detecting CXX compile features
[WARNING] -- Detecting CXX compile features - done
[WARNING] -- Looking for pthread.h
[WARNING] -- Looking for pthread.h - found
[WARNING] -- Performing Test CMAKE_HAVE_LIBC_PTHREAD
[WARNING] -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
[WARNING] -- Looking for pthread_create in pthreads
[WARNING] -- Looking for pthread_create in pthreads - not found
[WARNING] -- Looking for pthread_create in pthread
[WARNING] -- Looking for pthread_create in pthread - found
[WARNING] -- Found Threads: TRUE  
[WARNING] -- Found OpenSSL: /usr/lib64/libcrypto.so (found version "1.1.1k")  
[WARNING] -- Checking for module 'libtirpc'
[WARNING] --   Package 'libtirpc', required by 'virtual:world', not found
[WARNING] -- Looking for dlopen in dl
[WARNING] -- Looking for dlopen in dl - found
[WARNING] -- Configuring done
[WARNING] CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
[WARNING] Please set them or make sure they are set and tested correctly in the CMake files:
[WARNING] TIRPC_INCLUDE_DIRS
[WARNING]    used as include directory in directory /data/builder/hadoop-3.3.6-src/hadoop-tools/hadoop-pipes/src
[WARNING]    used as include directory in directory /data/builder/hadoop-3.3.6-src/hadoop-tools/hadoop-pipes/src
[WARNING]    used as include directory in directory /data/builder/hadoop-3.3.6-src/hadoop-tools/hadoop-pipes/src
[WARNING]    used as include directory in directory /data/builder/hadoop-3.3.6-src/hadoop-tools/hadoop-pipes/src
[WARNING]    used as include directory in directory /data/builder/hadoop-3.3.6-src/hadoop-tools/hadoop-pipes/src
[WARNING]    used as include directory in directory /data/builder/hadoop-3.3.6-src/hadoop-tools/hadoop-pipes/src
[WARNING]    used as include directory in directory /data/builder/hadoop-3.3.6-src/hadoop-tools/hadoop-pipes/src
[WARNING]    used as include directory in directory /data/builder/hadoop-3.3.6-src/hadoop-tools/hadoop-pipes/src
[WARNING] 
[WARNING] CMake Error in CMakeLists.txt:
[WARNING]   Found relative path while evaluating include directories of "hadooppipes":
[WARNING] 
[WARNING]     "TIRPC_INCLUDE_DIRS-NOTFOUND"
[WARNING] 
[WARNING] 
[WARNING] 
[WARNING] CMake Error in CMakeLists.txt:
[WARNING]   Found relative path while evaluating include directories of "hadooputils":
[WARNING] 
[WARNING]     "TIRPC_INCLUDE_DIRS-NOTFOUND"
[WARNING] 
[WARNING] 
[WARNING] 
[WARNING] CMake Error in CMakeLists.txt:
[WARNING]   Found relative path while evaluating include directories of
[WARNING]   "wordcount-simple":
[WARNING] 
[WARNING]     "TIRPC_INCLUDE_DIRS-NOTFOUND"
[WARNING] 
[WARNING] 
[WARNING] 
[WARNING] CMake Error in CMakeLists.txt:
[WARNING]   Found relative path while evaluating include directories of "pipes-sort":
[WARNING] 
[WARNING]     "TIRPC_INCLUDE_DIRS-NOTFOUND"
[WARNING] 
[WARNING] 
[WARNING] 
[WARNING] CMake Error in CMakeLists.txt:
[WARNING]   Found relative path while evaluating include directories of
[WARNING]   "wordcount-nopipe":
[WARNING] 
[WARNING]     "TIRPC_INCLUDE_DIRS-NOTFOUND"
[WARNING] 
[WARNING] 
[WARNING] 
[WARNING] CMake Error in CMakeLists.txt:
[WARNING]   Found relative path while evaluating include directories of
[WARNING]   "wordcount-part":
[WARNING] 
[WARNING]     "TIRPC_INCLUDE_DIRS-NOTFOUND"
[WARNING] 
[WARNING] 
[WARNING] 
[WARNING] -- Generating done
[WARNING] CMake Generate step failed.  Build files cannot be regenerated correctly.
...
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  06:33 min
[INFO] Finished at: 2023-09-03T14:05:29+08:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.hadoop:hadoop-maven-plugins:3.3.6:cmake-compile (cmake-compile) on project hadoop-pipes: CMake failed with error code 1 -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.hadoop:hadoop-maven-plugins:3.3.6:cmake-compile (cmake-compile) on project hadoop-pipes: CMake failed with error code 1
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:333)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:316)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:212)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:174)
    at org.apache.maven.lifecycle.internal.MojoExecutor.access$000 (MojoExecutor.java:75)
    at org.apache.maven.lifecycle.internal.MojoExecutor$1.run (MojoExecutor.java:162)
    at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute (DefaultMojosExecutionStrategy.java:39)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:159)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:105)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:73)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:53)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:118)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:261)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:173)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:101)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:906)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:283)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:206)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:283)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:226)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:407)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:348)
Caused by: org.apache.maven.plugin.MojoExecutionException: CMake failed with error code 1
    at org.apache.hadoop.maven.plugin.cmakebuilder.CompileMojo.runCMake (CompileMojo.java:172)
    at org.apache.hadoop.maven.plugin.cmakebuilder.CompileMojo.execute (CompileMojo.java:97)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:126)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:328)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:316)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:212)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:174)
    at org.apache.maven.lifecycle.internal.MojoExecutor.access$000 (MojoExecutor.java:75)
    at org.apache.maven.lifecycle.internal.MojoExecutor$1.run (MojoExecutor.java:162)
    at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute (DefaultMojosExecutionStrategy.java:39)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:159)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:105)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:73)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:53)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:118)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:261)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:173)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:101)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:906)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:283)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:206)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:283)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:226)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:407)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:348)
[ERROR] 
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <args> -rf :hadoop-pipes

实际错误为：

[WARNING] --   Package 'libtirpc', required by 'virtual:world', not found

这是因为没有按照要求安装依赖，导致缺失 libtirpc-devel ，手动补充安装。

sudo dnf install -y libtirpc-devel

b) 编译报错 org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal com.github.eirslett:frontend-maven-plugin:1.11.2:yarn (yarn install) on project hadoop-yarn-applications-catalog-webapp: Failed to run task 详细如下：

[INFO] testFailureIgnore property is ignored in non test phases
[INFO] Installing node version v12.22.1
[DEBUG] Creating install directory /data/builder/hadoop-3.3.6-src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/target/node
[DEBUG] Creating temporary directory /data/builder/hadoop-3.3.6-src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/target/node/tmp
[INFO] Downloading https://nodejs.org/dist/v12.22.1/node-v12.22.1-linux-x64.tar.gz to /home/kane/.m2/repository/com/github/eirslett/node/12.22.1/node-12.22.1-linux-x64.tar.gz
[INFO] No proxies configured
[INFO] No proxy was configured, downloading directly
[INFO] Unpacking /home/kane/.m2/repository/com/github/eirslett/node/12.22.1/node-12.22.1-linux-x64.tar.gz into /data/builder/hadoop-3.3.6-src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/target/node/tmp
[INFO] Copying node binary from /data/builder/hadoop-3.3.6-src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/target/node/tmp/node-v12.22.1-linux-x64/bin/node to /data/builder/hadoop-3.3.6-src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/target/node/node
[DEBUG] Deleting temporary directory /data/builder/hadoop-3.3.6-src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/target/node/tmp
[INFO] Installed node locally.
[INFO] Installing Yarn version v1.22.5
[INFO] Downloading https://github.com/yarnpkg/yarn/releases/download/v1.22.5/yarn-v1.22.5.tar.gz to /home/kane/.m2/repository/com/github/eirslett/yarn/1.22.5/yarn-1.22.5.tar.gz
[INFO] No proxies configured
[INFO] No proxy was configured, downloading directly
[DEBUG] Creating install directory /data/builder/hadoop-3.3.6-src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/target/node/yarn
[INFO] Unpacking /home/kane/.m2/repository/com/github/eirslett/yarn/1.22.5/yarn-1.22.5.tar.gz into /data/builder/hadoop-3.3.6-src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/target/node/yarn
[DEBUG] Yarn root directory not found, checking for yarn-v1.22.5
[INFO] Installed Yarn locally.
[INFO] 
[INFO] --- frontend:1.11.2:yarn (yarn install) @ hadoop-yarn-applications-catalog-webapp ---
[DEBUG] Loading mojo com.github.eirslett:frontend-maven-plugin:1.11.2:yarn from plugin realm ClassRealm[plugin>com.github.eirslett:frontend-maven-plugin:1.11.2, parent: sun.misc.Launcher$AppClassLoader@7852e922]
[DEBUG] Configuring mojo execution 'com.github.eirslett:frontend-maven-plugin:1.11.2:yarn:yarn install' with basic configurator -->
[DEBUG]   (f) project = MavenProject: org.apache.hadoop:hadoop-yarn-applications-catalog-webapp:3.3.6 @ /data/builder/hadoop-3.3.6-src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/pom.xml
[DEBUG]   (f) repositorySystemSession = org.eclipse.aether.DefaultRepositorySystemSession@6e668235
[DEBUG]   (f) session = org.apache.maven.execution.MavenSession@350d3f4d
[DEBUG]   (f) skipTests = true
[DEBUG]   (f) testFailureIgnore = true
[DEBUG]   (f) workingDirectory = /data/builder/hadoop-3.3.6-src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/target
[DEBUG]   (f) yarnInheritsProxyConfigFromMaven = true
[DEBUG]   (f) execution = com.github.eirslett:frontend-maven-plugin:1.11.2:yarn {execution: yarn install}
[DEBUG] -- end configuration --
[INFO] testFailureIgnore property is ignored in non test phases
[INFO] Running 'yarn ' in /data/builder/hadoop-3.3.6-src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/target
[DEBUG] Executing command line [/data/builder/hadoop-3.3.6-src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp/target/node/yarn/dist/bin/yarn]
[INFO] yarn install v1.22.5
[INFO] info No lockfile found.
[INFO] [1/4] Resolving packages...
[INFO] warning angular-route@1.6.10: For the actively supported Angular, see https://www.npmjs.com/package/@angular/core. AngularJS support has officially ended. For extended AngularJS support options, see https://goo.gle/angularjs-path-forward.
[INFO] warning angular@1.6.10: For the actively supported Angular, see https://www.npmjs.com/package/@angular/core. AngularJS support has officially ended. For extended AngularJS support options, see https://goo.gle/angularjs-path-forward.
[INFO] [2/4] Fetching packages...
[INFO] error triple-beam@1.4.1: The engine "node" is incompatible with this module. Expected version ">= 14.0.0". Got "12.22.1"
[INFO] error Found incompatible module.
[INFO] info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Apache Hadoop Main 3.3.6:
[INFO] 
[INFO] Apache Hadoop Main ................................. SUCCESS [  1.219 s]
[INFO] Apache Hadoop Build Tools .......................... SUCCESS [  0.695 s]
[INFO] Apache Hadoop Project POM .......................... SUCCESS [  1.193 s]
[INFO] Apache Hadoop Annotations .......................... SUCCESS [  1.610 s]
[INFO] Apache Hadoop Assemblies ........................... SUCCESS [  0.084 s]
[INFO] Apache Hadoop Project Dist POM ..................... SUCCESS [  1.033 s]
[INFO] Apache Hadoop Maven Plugins ........................ SUCCESS [  2.419 s]
[INFO] Apache Hadoop MiniKDC .............................. SUCCESS [  0.910 s]
[INFO] Apache Hadoop Auth ................................. SUCCESS [  4.007 s]
[INFO] Apache Hadoop Auth Examples ........................ SUCCESS [  1.632 s]
[INFO] Apache Hadoop Common ............................... SUCCESS [ 21.683 s]
[INFO] Apache Hadoop NFS .................................. SUCCESS [  3.751 s]
[INFO] Apache Hadoop KMS .................................. SUCCESS [  3.017 s]
[INFO] Apache Hadoop Registry ............................. SUCCESS [  3.450 s]
[INFO] Apache Hadoop Common Project ....................... SUCCESS [  0.091 s]
[INFO] Apache Hadoop HDFS Client .......................... SUCCESS [ 16.213 s]
[INFO] Apache Hadoop HDFS ................................. SUCCESS [ 41.895 s]
[INFO] Apache Hadoop HDFS Native Client ................... SUCCESS [01:49 min]
[INFO] Apache Hadoop HttpFS ............................... SUCCESS [  4.328 s]
[INFO] Apache Hadoop HDFS-NFS ............................. SUCCESS [  3.556 s]
[INFO] Apache Hadoop HDFS-RBF ............................. SUCCESS [ 19.910 s]
[INFO] Apache Hadoop HDFS Project ......................... SUCCESS [  0.045 s]
[INFO] Apache Hadoop YARN ................................. SUCCESS [  0.051 s]
[INFO] Apache Hadoop YARN API ............................. SUCCESS [  8.846 s]
[INFO] Apache Hadoop YARN Common .......................... SUCCESS [ 15.550 s]
[INFO] Apache Hadoop YARN Server .......................... SUCCESS [  0.040 s]
[INFO] Apache Hadoop YARN Server Common ................... SUCCESS [  6.592 s]
[INFO] Apache Hadoop YARN NodeManager ..................... SUCCESS [ 26.872 s]
[INFO] Apache Hadoop YARN Web Proxy ....................... SUCCESS [  3.021 s]
[INFO] Apache Hadoop YARN ApplicationHistoryService ....... SUCCESS [  3.793 s]
[INFO] Apache Hadoop YARN Timeline Service ................ SUCCESS [  3.344 s]
[INFO] Apache Hadoop YARN ResourceManager ................. SUCCESS [ 13.319 s]
[INFO] Apache Hadoop YARN Server Tests .................... SUCCESS [  2.213 s]
[INFO] Apache Hadoop YARN Client .......................... SUCCESS [  4.149 s]
[INFO] Apache Hadoop YARN SharedCacheManager .............. SUCCESS [  2.985 s]
[INFO] Apache Hadoop YARN Timeline Plugin Storage ......... SUCCESS [  2.867 s]
[INFO] Apache Hadoop YARN TimelineService HBase Backend ... SUCCESS [  0.032 s]
[INFO] Apache Hadoop YARN TimelineService HBase Common .... SUCCESS [  3.122 s]
[INFO] Apache Hadoop YARN TimelineService HBase Client .... SUCCESS [  4.139 s]
[INFO] Apache Hadoop YARN TimelineService HBase Servers ... SUCCESS [  0.036 s]
[INFO] Apache Hadoop YARN TimelineService HBase Server 1.2  SUCCESS [  3.857 s]
[INFO] Apache Hadoop YARN TimelineService HBase tests ..... SUCCESS [  2.778 s]
[INFO] Apache Hadoop YARN Router .......................... SUCCESS [  3.562 s]
[INFO] Apache Hadoop YARN TimelineService DocumentStore ... SUCCESS [  2.745 s]
[INFO] Apache Hadoop YARN Applications .................... SUCCESS [  0.028 s]
[INFO] Apache Hadoop YARN DistributedShell ................ SUCCESS [  3.472 s]
[INFO] Apache Hadoop YARN Unmanaged Am Launcher ........... SUCCESS [  2.352 s]
[INFO] Apache Hadoop MapReduce Client ..................... SUCCESS [  1.616 s]
[INFO] Apache Hadoop MapReduce Core ....................... SUCCESS [  4.119 s]
[INFO] Apache Hadoop MapReduce Common ..................... SUCCESS [  4.410 s]
[INFO] Apache Hadoop MapReduce Shuffle .................... SUCCESS [  3.049 s]
[INFO] Apache Hadoop MapReduce App ........................ SUCCESS [  5.465 s]
[INFO] Apache Hadoop MapReduce HistoryServer .............. SUCCESS [  4.355 s]
[INFO] Apache Hadoop MapReduce JobClient .................. SUCCESS [  4.536 s]
[INFO] Apache Hadoop Mini-Cluster ......................... SUCCESS [  2.550 s]
[INFO] Apache Hadoop YARN Services ........................ SUCCESS [  0.037 s]
[INFO] Apache Hadoop YARN Services Core ................... SUCCESS [  3.400 s]
[INFO] Apache Hadoop YARN Services API .................... SUCCESS [  2.833 s]
[INFO] Apache Hadoop YARN Application Catalog ............. SUCCESS [  0.033 s]
[INFO] Apache Hadoop YARN Application Catalog Webapp ...... FAILURE [ 40.802 s]
[INFO] Apache Hadoop YARN Application Catalog Docker Image  SKIPPED
[INFO] Apache Hadoop YARN Application MaWo ................ SKIPPED
[INFO] Apache Hadoop YARN Application MaWo Core ........... SKIPPED
[INFO] Apache Hadoop YARN Site ............................ SKIPPED
[INFO] Apache Hadoop YARN Registry ........................ SKIPPED
[INFO] Apache Hadoop YARN UI .............................. SKIPPED
[INFO] Apache Hadoop YARN CSI ............................. SKIPPED
[INFO] Apache Hadoop YARN Project ......................... SKIPPED
[INFO] Apache Hadoop MapReduce HistoryServer Plugins ...... SKIPPED
[INFO] Apache Hadoop MapReduce NativeTask ................. SKIPPED
[INFO] Apache Hadoop MapReduce Uploader ................... SKIPPED
[INFO] Apache Hadoop MapReduce Examples ................... SKIPPED
[INFO] Apache Hadoop MapReduce ............................ SKIPPED
[INFO] Apache Hadoop MapReduce Streaming .................. SKIPPED
[INFO] Apache Hadoop Distributed Copy ..................... SKIPPED
[INFO] Apache Hadoop Client Aggregator .................... SKIPPED
[INFO] Apache Hadoop Dynamometer Workload Simulator ....... SKIPPED
[INFO] Apache Hadoop Dynamometer Cluster Simulator ........ SKIPPED
[INFO] Apache Hadoop Dynamometer Block Listing Generator .. SKIPPED
[INFO] Apache Hadoop Dynamometer Dist ..................... SKIPPED
[INFO] Apache Hadoop Dynamometer .......................... SKIPPED
[INFO] Apache Hadoop Archives ............................. SKIPPED
[INFO] Apache Hadoop Archive Logs ......................... SKIPPED
[INFO] Apache Hadoop Rumen ................................ SKIPPED
[INFO] Apache Hadoop Gridmix .............................. SKIPPED
[INFO] Apache Hadoop Data Join ............................ SKIPPED
[INFO] Apache Hadoop Extras ............................... SKIPPED
[INFO] Apache Hadoop Pipes ................................ SKIPPED
[INFO] Apache Hadoop Amazon Web Services support .......... SKIPPED
[INFO] Apache Hadoop Kafka Library support ................ SKIPPED
[INFO] Apache Hadoop Azure support ........................ SKIPPED
[INFO] Apache Hadoop Aliyun OSS support ................... SKIPPED
[INFO] Apache Hadoop Scheduler Load Simulator ............. SKIPPED
[INFO] Apache Hadoop Resource Estimator Service ........... SKIPPED
[INFO] Apache Hadoop Azure Data Lake support .............. SKIPPED
[INFO] Apache Hadoop Image Generation Tool ................ SKIPPED
[INFO] Apache Hadoop Tools Dist ........................... SKIPPED
[INFO] Apache Hadoop OpenStack support .................... SKIPPED
[INFO] Apache Hadoop Common Benchmark ..................... SKIPPED
[INFO] Apache Hadoop Tools ................................ SKIPPED
[INFO] Apache Hadoop Client API ........................... SKIPPED
[INFO] Apache Hadoop Client Runtime ....................... SKIPPED
[INFO] Apache Hadoop Client Packaging Invariants .......... SKIPPED
[INFO] Apache Hadoop Client Test Minicluster .............. SKIPPED
[INFO] Apache Hadoop Client Packaging Invariants for Test . SKIPPED
[INFO] Apache Hadoop Client Packaging Integration Tests ... SKIPPED
[INFO] Apache Hadoop Distribution ......................... SKIPPED
[INFO] Apache Hadoop Client Modules ....................... SKIPPED
[INFO] Apache Hadoop Tencent COS Support .................. SKIPPED
[INFO] Apache Hadoop Cloud Storage ........................ SKIPPED
[INFO] Apache Hadoop Cloud Storage Project ................ SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  07:20 min
[INFO] Finished at: 2023-09-03T15:54:02+08:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal com.github.eirslett:frontend-maven-plugin:1.11.2:yarn (yarn install) on project hadoop-yarn-applications-catalog-webapp: Failed to run task: 'yarn ' failed. org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal com.github.eirslett:frontend-maven-plugin:1.11.2:yarn (yarn install) on project hadoop-yarn-applications-catalog-webapp: Failed to run task
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:333)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:316)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:212)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:174)
    at org.apache.maven.lifecycle.internal.MojoExecutor.access$000 (MojoExecutor.java:75)
    at org.apache.maven.lifecycle.internal.MojoExecutor$1.run (MojoExecutor.java:162)
    at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute (DefaultMojosExecutionStrategy.java:39)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:159)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:105)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:73)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:53)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:118)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:261)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:173)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:101)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:906)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:283)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:206)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:283)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:226)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:407)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:348)
Caused by: org.apache.maven.plugin.MojoFailureException: Failed to run task
    at com.github.eirslett.maven.plugins.frontend.mojo.AbstractFrontendMojo.execute (AbstractFrontendMojo.java:100)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:126)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:328)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:316)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:212)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:174)
    at org.apache.maven.lifecycle.internal.MojoExecutor.access$000 (MojoExecutor.java:75)
    at org.apache.maven.lifecycle.internal.MojoExecutor$1.run (MojoExecutor.java:162)
    at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute (DefaultMojosExecutionStrategy.java:39)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:159)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:105)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:73)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:53)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:118)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:261)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:173)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:101)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:906)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:283)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:206)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:283)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:226)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:407)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:348)
Caused by: com.github.eirslett.maven.plugins.frontend.lib.TaskRunnerException: 'yarn ' failed.
    at com.github.eirslett.maven.plugins.frontend.lib.YarnTaskExecutor.execute (YarnTaskExecutor.java:64)
    at com.github.eirslett.maven.plugins.frontend.mojo.YarnMojo.execute (YarnMojo.java:65)
    at com.github.eirslett.maven.plugins.frontend.mojo.AbstractFrontendMojo.execute (AbstractFrontendMojo.java:94)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:126)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:328)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:316)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:212)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:174)
    at org.apache.maven.lifecycle.internal.MojoExecutor.access$000 (MojoExecutor.java:75)
    at org.apache.maven.lifecycle.internal.MojoExecutor$1.run (MojoExecutor.java:162)
    at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute (DefaultMojosExecutionStrategy.java:39)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:159)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:105)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:73)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:53)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:118)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:261)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:173)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:101)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:906)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:283)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:206)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:283)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:226)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:407)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:348)
Caused by: com.github.eirslett.maven.plugins.frontend.lib.ProcessExecutionException: org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1)
    at com.github.eirslett.maven.plugins.frontend.lib.ProcessExecutor.execute (ProcessExecutor.java:82)
    at com.github.eirslett.maven.plugins.frontend.lib.ProcessExecutor.executeAndRedirectOutput (ProcessExecutor.java:64)
    at com.github.eirslett.maven.plugins.frontend.lib.YarnExecutor.executeAndRedirectOutput (YarnExecutor.java:28)
    at com.github.eirslett.maven.plugins.frontend.lib.YarnTaskExecutor.execute (YarnTaskExecutor.java:58)
    at com.github.eirslett.maven.plugins.frontend.mojo.YarnMojo.execute (YarnMojo.java:65)
    at com.github.eirslett.maven.plugins.frontend.mojo.AbstractFrontendMojo.execute (AbstractFrontendMojo.java:94)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:126)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:328)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:316)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:212)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:174)
    at org.apache.maven.lifecycle.internal.MojoExecutor.access$000 (MojoExecutor.java:75)
    at org.apache.maven.lifecycle.internal.MojoExecutor$1.run (MojoExecutor.java:162)
    at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute (DefaultMojosExecutionStrategy.java:39)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:159)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:105)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:73)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:53)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:118)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:261)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:173)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:101)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:906)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:283)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:206)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:283)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:226)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:407)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:348)
Caused by: org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1)
    at org.apache.commons.exec.DefaultExecutor.executeInternal (DefaultExecutor.java:404)
    at org.apache.commons.exec.DefaultExecutor.execute (DefaultExecutor.java:166)
    at com.github.eirslett.maven.plugins.frontend.lib.ProcessExecutor.execute (ProcessExecutor.java:74)
    at com.github.eirslett.maven.plugins.frontend.lib.ProcessExecutor.executeAndRedirectOutput (ProcessExecutor.java:64)
    at com.github.eirslett.maven.plugins.frontend.lib.YarnExecutor.executeAndRedirectOutput (YarnExecutor.java:28)
    at com.github.eirslett.maven.plugins.frontend.lib.YarnTaskExecutor.execute (YarnTaskExecutor.java:58)
    at com.github.eirslett.maven.plugins.frontend.mojo.YarnMojo.execute (YarnMojo.java:65)
    at com.github.eirslett.maven.plugins.frontend.mojo.AbstractFrontendMojo.execute (AbstractFrontendMojo.java:94)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:126)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:328)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:316)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:212)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:174)
    at org.apache.maven.lifecycle.internal.MojoExecutor.access$000 (MojoExecutor.java:75)
    at org.apache.maven.lifecycle.internal.MojoExecutor$1.run (MojoExecutor.java:162)
    at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute (DefaultMojosExecutionStrategy.java:39)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:159)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:105)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:73)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:53)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:118)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:261)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:173)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:101)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:906)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:283)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:206)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:283)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:226)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:407)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:348)
[ERROR] 
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <args> -rf :hadoop-yarn-applications-catalog-webapp

关键信息只有一行

[INFO] error triple-beam@1.4.1: The engine "node" is incompatible with this module. Expected version ">= 14.0.0". Got "12.22.1"

其中的部分组件不兼容旧版的 Nodejs，需要修改配置 hadoop-project/pom.xml 中找到 nodejs.version 和 yarnpkg.version 字段，改为如下的版本，可用版本可在各自官网 node.js 和 yarn 中搜索。

  <properties>
    ...
    <nodejs.version>v14.21.3</nodejs.version>
    <yarnpkg.version>v1.22.19</yarnpkg.version>
    ...
  </properties>

附录

参考链接

Native Libraries Guide - Hadoop Official Docs

本文由柒创作，采用知识共享署名4.0 国际许可协议进行许可。
转载本站文章前请注明出处，文章作者保留所有权限。
最后编辑时间： 2023-10-06 18:37 PM