update to 4.0.0 release by daltonbohning · Pull Request #14 · daos-stack/ior

daltonbohning · 2024-01-12T18:57:27Z

update to 4.0.0 release
update packaging

Signed-off-by: Dalton Bohning <dalton.bohning@intel.com>

daltonbohning · 2024-01-16T18:41:20Z

ior.spec

 %endif
-Patch3: daos-configure.patch
+# patch configure.ac
+Patch3: https://github.com/hpc/ior/commit/38064419cbe959cb538695e51b2bc2a91d6971f7.patch


We could also just update to 4.0.0 + patch tip of main. Which is what the spec does currently. I.e. 3.3.0..d3574d536643475269d37211e283b49ebd6732d7.patch

We could, at the risk of introducing post-4.0.0 landings that could be troublesome. I think what we are doing here is better and more idiomatic of software packaging.

daltonbohning · 2024-01-16T18:41:55Z

ior.spec

 Summary:	IOR-HPC

-License:	GPL
+License:    GPL-2.0-only


To resolve invalid-license lint

ior.spec

daltonbohning · 2024-01-16T18:46:58Z

@brianjmurrell Any advice on how to fix this lint warning for leap 15.5?
https://build.hpdd.intel.com/blue/rest/organizations/jenkins/pipelines/daos-stack/pipelines/ior/branches/PR-14/runs/8/nodes/206/log/?start=0

RPM Lint found warnings:
ior.x86_64: W: position-independent-executable-suggested /usr/bin/ior
ior.x86_64: W: position-independent-executable-suggested /usr/bin/md-workbench
ior.x86_64: W: position-independent-executable-suggested /usr/bin/mdtest

I thought this was handled by

ior/ior.spec

Lines 61 to 62 in ecea31d

    
           export CFLAGS="$RPM_OPT_FLAGS -fno-strict-aliasing -fPIC" 
        
           export CXXFLAGS="$RPM_OPT_FLAGS -fno-strict-aliasing -fPIC"

brianjmurrell · 2024-01-16T21:15:12Z

I thought this was handled by

ior/ior.spec

Lines 61 to 62 in ecea31d

export CFLAGS="$RPM_OPT_FLAGS -fno-strict-aliasing -fPIC"

export CXXFLAGS="$RPM_OPT_FLAGS -fno-strict-aliasing -fPIC"

I would think so too. As long at the ior build system was not overriding or ignoring those in some way. Does the build log show that they are being used? You might have to add a V=1 to the make arguments to get verbose command lines.

They in fact shouldn't even be necessary as hardened_build is default in EL > 7. Ahhh. But maybe necessary still for Leap. If so, we ought to guard that so that it doesn't override the goodness that RHEL bakes into their RPM build macros.

daltonbohning · 2024-01-16T23:30:43Z

Does the build log show that they are being used?

For EL9, I see -fPIC in the build commands, but for Leap I do not. I'll try with -V.

Signed-off-by: Dalton Bohning <dalton.bohning@intel.com>

daltonbohning · 2024-01-16T23:54:53Z

It looks like it's using it?
https://build.hpdd.intel.com/job/daos-stack/job/ior/job/PR-14/9/artifact/artifacts/leap15/build.log/*view*/

Making all in .
mpicc -DHAVE_CONFIG_H -I.      -Icheck/include -Icheck/include -I/usr/include  -O2 -g -m64 -fmessage-length=0 -D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables -fasynchronous-unwind-tables -fno-strict-aliasing -fPIC -c -o md_workbench-md-workbench-main.o `test -f 'md-workbench-main.c' || echo './'`md-workbench-main.c
...

Specifically, for the mdtest executable

mpicc  -O2 -g -m64 -fmessage-length=0 -D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables -fasynchronous-unwind-tables -fno-strict-aliasing -fPIC     -Lcheck/lib64 -Wl,--enable-new-dtags -Wl,-rpath=check/lib64 -Lcheck/lib64 -Wl,--enable-new-dtags -Wl,-rpath=check/lib64 -L/usr/lib64 -Wl,--enable-new-dtags -Wl,-rpath=/usr/lib64 -o mdtest mdtest-mdtest-main.o mdtest-aiori.o mdtest-aiori-DUMMY.o   mdtest-aiori-HDF5.o  mdtest-aiori-MPIIO.o  mdtest-aiori-MMAP.o

But lint complains

RPM Lint found warnings:
ior.x86_64: W: position-independent-executable-suggested /usr/bin/ior
ior.x86_64: W: position-independent-executable-suggested /usr/bin/md-workbench
ior.x86_64: W: position-independent-executable-suggested /usr/bin/mdtest

daltonbohning · 2024-01-18T21:54:47Z

Ah. Maybe it's -fPIC (position-independent-code) vs -fPIE (position-independent-executable)

daltonbohning · 2024-01-18T21:57:13Z

It seems -fPIC should be used when creating the .so, and -fPIE when creating the executables

daltonbohning · 2024-01-18T21:59:20Z

Looking at the EL9 log for the mdtest executable, -fPIC is used but not -fPIE. So I'm confused
https://build.hpdd.intel.com/job/daos-stack/job/ior/job/PR-14/9/artifact/artifacts/el9/build.log

mpicc  -O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -m64 -march=x86-64-v2 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fno-strict-aliasing -fPIC     -Wl,-z,relro -Wl,--as-needed  -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -Lcheck/lib64 -Wl,--enable-new-dtags -Wl,-rpath=check/lib64 -Lcheck/lib64 -Wl,--enable-new-dtags -Wl,-rpath=check/lib64 -L/usr/lib64 -Wl,--enable-new-dtags -Wl,-rpath=/usr/lib64 -o mdtest mdtest-mdtest-main.o mdtest-aiori.o mdtest-aiori-DUMMY.o   mdtest-aiori-HDF5.o  mdtest-aiori-MPIIO.o  mdtest-aiori-MMAP.o mdtest-aiori-POSIX.o     mdtest-aiori-DFS.o     libaiori.a    -lhdf5 -lz       -lgurt -ldaos_common -ldaos -ldfs -luuid      -ldfs -ldaos -luuid -lgurt -lhdf5 -lm

Signed-off-by: Dalton Bohning <dalton.bohning@intel.com>

ior.spec

brianjmurrell · 2024-01-22T15:12:14Z

These:

ior.x86_64: W: position-independent-executable-suggested /usr/bin/ior
ior.x86_64: W: position-independent-executable-suggested /usr/bin/md-workbench
ior.x86_64: W: position-independent-executable-suggested /usr/bin/mdtest

are really odd given that we can see that the link (and all of the object compilations) are using the hardening flags:

mpicc  -O2 -g -m64 -fmessage-length=0 -D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables -fasynchronous-unwind-tables -fno-strict-aliasing -fPIC -fPIE     -Lcheck/lib64 -Wl,--enable-new-dtags -Wl,-rpath=check/lib64 -Lcheck/lib64 -Wl,--enable-new-dtags -Wl,-rpath=check/lib64 -L/usr/lib64 -Wl,--enable-new-dtags -Wl,-rpath=/usr/lib64 -o mdtest mdtest-mdtest-main.o mdtest-aiori.o mdtest-aiori-DUMMY.o   mdtest-aiori-HDF5.o  mdtest-aiori-MPIIO.o  mdtest-aiori-MMAP.o mdtest-aiori-POSIX.o     mdtest-aiori-DFS.o     libaiori.a    -lhdf5 -lz       -lgurt -ldaos_common -ldaos -ldfs -luuid      -ldfs -ldaos -luuid -lgurt -lhdf5 -lm 
mpicc  -O2 -g -m64 -fmessage-length=0 -D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables -fasynchronous-unwind-tables -fno-strict-aliasing -fPIC -fPIE     -Lcheck/lib64 -Wl,--enable-new-dtags -Wl,-rpath=check/lib64 -Lcheck/lib64 -Wl,--enable-new-dtags -Wl,-rpath=check/lib64 -L/usr/lib64 -Wl,--enable-new-dtags -Wl,-rpath=/usr/lib64 -o ior ior-ior-main.o ior-aiori.o ior-aiori-DUMMY.o   ior-aiori-HDF5.o  ior-aiori-MPIIO.o  ior-aiori-MMAP.o ior-aiori-POSIX.o     ior-aiori-DFS.o     libaiori.a    -lhdf5 -lz       -lgurt -ldaos_common -ldaos -ldfs -luuid      -ldfs -ldaos -luuid -lgurt -lhdf5 -lm 
mpicc  -O2 -g -m64 -fmessage-length=0 -D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables -fasynchronous-unwind-tables -fno-strict-aliasing -fPIC -fPIE     -Lcheck/lib64 -Wl,--enable-new-dtags -Wl,-rpath=check/lib64 -Lcheck/lib64 -Wl,--enable-new-dtags -Wl,-rpath=check/lib64 -L/usr/lib64 -Wl,--enable-new-dtags -Wl,-rpath=/usr/lib64 -o md-workbench md_workbench-md-workbench-main.o md_workbench-aiori.o md_workbench-aiori-DUMMY.o   md_workbench-aiori-HDF5.o  md_workbench-aiori-MPIIO.o  md_workbench-aiori-MMAP.o md_workbench-aiori-POSIX.o     md_workbench-aiori-DFS.o     libaiori.a    -lhdf5 -lz       -lgurt -ldaos_common -ldaos -ldfs -luuid      -ldfs -ldaos -luuid -lgurt -lhdf5 -lm

This is the link command from EL8:

mpicc  -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fno-strict-aliasing -fPIC -fPIE     -Wl,-z,relro  -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -Lcheck/lib64 -Wl,--enable-new-dtags -Wl,-rpath=check/lib64 -Lcheck/lib64 -Wl,--enable-new-dtags -Wl,-rpath=check/lib64 -L/usr/lib64 -Wl,--enable-new-dtags -Wl,-rpath=/usr/lib64 -o ior ior-ior-main.o ior-aiori.o ior-aiori-DUMMY.o   ior-aiori-HDF5.o  ior-aiori-MPIIO.o  ior-aiori-MMAP.o ior-aiori-POSIX.o     ior-aiori-DFS.o     libaiori.a    -lhdf5 -lz       -lgurt -ldaos_common -ldaos -ldfs -luuid      -ldfs -ldaos -luuid -lgurt -lhdf5 -lm 
mpicc  -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fno-strict-aliasing -fPIC -fPIE     -Wl,-z,relro  -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -Lcheck/lib64 -Wl,--enable-new-dtags -Wl,-rpath=check/lib64 -Lcheck/lib64 -Wl,--enable-new-dtags -Wl,-rpath=check/lib64 -L/usr/lib64 -Wl,--enable-new-dtags -Wl,-rpath=/usr/lib64 -o mdtest mdtest-mdtest-main.o mdtest-aiori.o mdtest-aiori-DUMMY.o   mdtest-aiori-HDF5.o  mdtest-aiori-MPIIO.o  mdtest-aiori-MMAP.o mdtest-aiori-POSIX.o     mdtest-aiori-DFS.o     libaiori.a    -lhdf5 -lz       -lgurt -ldaos_common -ldaos -ldfs -luuid      -ldfs -ldaos -luuid -lgurt -lhdf5 -lm 
mpicc  -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fno-strict-aliasing -fPIC -fPIE     -Wl,-z,relro  -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -Lcheck/lib64 -Wl,--enable-new-dtags -Wl,-rpath=check/lib64 -Lcheck/lib64 -Wl,--enable-new-dtags -Wl,-rpath=check/lib64 -L/usr/lib64 -Wl,--enable-new-dtags -Wl,-rpath=/usr/lib64 -o md-workbench md_workbench-md-workbench-main.o md_workbench-aiori.o md_workbench-aiori-DUMMY.o   md_workbench-aiori-HDF5.o  md_workbench-aiori-MPIIO.o  md_workbench-aiori-MMAP.o md_workbench-aiori-POSIX.o     md_workbench-aiori-DFS.o     libaiori.a    -lhdf5 -lz       -lgurt -ldaos_common -ldaos -ldfs -luuid      -ldfs -ldaos -luuid -lgurt -lhdf5 -lm

RH puts all of their hardening in gcc with this option -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 which is simply:

*cc1_options:
+ %{!r:%{!fpie:%{!fPIE:%{!fpic:%{!fPIC:%{!fno-pic:-fPIE}}}}}}

which we can is some flavour of -fPIE.

Signed-off-by: Dalton Bohning <dalton.bohning@intel.com>

update to 4.0.0 release

6b28adf

Signed-off-by: Dalton Bohning <dalton.bohning@intel.com>

daltonbohning self-assigned this Jan 12, 2024

daltonbohning requested a review from a team as a code owner January 12, 2024 18:57

daltonbohning marked this pull request as draft January 12, 2024 18:58

daltonbohning added 7 commits January 12, 2024 19:03

update packaging

fbc5681

Signed-off-by: Dalton Bohning <dalton.bohning@intel.com>

do not build centos7

24ecda7

Signed-off-by: Dalton Bohning <dalton.bohning@intel.com>

Remove BR: mercury-devel and use upstream patch

3edbdbc

Signed-off-by: Dalton Bohning <dalton.bohning@intel.com>

attempt to fix license linting

e4346a4

Signed-off-by: Dalton Bohning <dalton.bohning@intel.com>

attempt fix rpm lint file-not-in-%lang

ef4c220

Signed-off-by: Dalton Bohning <dalton.bohning@intel.com>

try --with-man

4864ccd

Signed-off-by: Dalton Bohning <dalton.bohning@intel.com>

move find_lang

ecea31d

Signed-off-by: Dalton Bohning <dalton.bohning@intel.com>

daltonbohning commented Jan 16, 2024

View reviewed changes

ior.spec

Summary: IOR-HPC

License: GPL

License: GPL-2.0-only

Copy link

Author

daltonbohning Jan 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To resolve invalid-license lint

daltonbohning commented Jan 16, 2024

View reviewed changes

ior.spec Show resolved Hide resolved

add V=1

062d5c4

Signed-off-by: Dalton Bohning <dalton.bohning@intel.com>

try both -fPIC and -fPIE for giggles

af3ed38

Signed-off-by: Dalton Bohning <dalton.bohning@intel.com>

brianjmurrell reviewed Jan 22, 2024

View reviewed changes

ior.spec Show resolved Hide resolved

brianjmurrell reviewed Jan 22, 2024

View reviewed changes

ior.spec Outdated Show resolved Hide resolved

daltonbohning added 4 commits February 28, 2024 23:04

update packaging again

1d60a7d

Signed-off-by: Dalton Bohning <dalton.bohning@intel.com>

update packaging again

30c6c52

Signed-off-by: Dalton Bohning <dalton.bohning@intel.com>

update changelog

f475c5d

Signed-off-by: Dalton Bohning <dalton.bohning@intel.com>

review update

199a099

Signed-off-by: Dalton Bohning <dalton.bohning@intel.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

update to 4.0.0 release#14

update to 4.0.0 release#14
daltonbohning wants to merge 14 commits intomasterfrom
dbohning/daos-14979

daltonbohning commented Jan 12, 2024 •

edited

Loading

Uh oh!

daltonbohning Jan 16, 2024

Uh oh!

brianjmurrell Jan 22, 2024

Uh oh!

daltonbohning Jan 16, 2024

Uh oh!

Uh oh!

daltonbohning commented Jan 16, 2024

Uh oh!

brianjmurrell commented Jan 16, 2024

Uh oh!

daltonbohning commented Jan 16, 2024

Uh oh!

daltonbohning commented Jan 16, 2024 •

edited

Loading

Uh oh!

daltonbohning commented Jan 18, 2024

Uh oh!

daltonbohning commented Jan 18, 2024

Uh oh!

daltonbohning commented Jan 18, 2024

Uh oh!

Uh oh!

Uh oh!

brianjmurrell commented Jan 22, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

daltonbohning commented Jan 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

daltonbohning Jan 16, 2024

Choose a reason for hiding this comment

Uh oh!

brianjmurrell Jan 22, 2024

Choose a reason for hiding this comment

Uh oh!

daltonbohning Jan 16, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!

daltonbohning commented Jan 16, 2024

Uh oh!

brianjmurrell commented Jan 16, 2024

Uh oh!

daltonbohning commented Jan 16, 2024

Uh oh!

daltonbohning commented Jan 16, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

daltonbohning commented Jan 18, 2024

Uh oh!

daltonbohning commented Jan 18, 2024

Uh oh!

daltonbohning commented Jan 18, 2024

Uh oh!

Uh oh!

Uh oh!

brianjmurrell commented Jan 22, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

daltonbohning commented Jan 12, 2024 •

edited

Loading

daltonbohning commented Jan 16, 2024 •

edited

Loading