Cannot build on macOS 10.15 (Catalina)

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Cannot build on macOS 10.15 (Catalina)

Eddy Hahn
Hi,

I got the latest source and build tools. 

1) autorecon -If. WORKED!

2) configure WORKED!

LIBUV_LIBS="-L$/dependencies/libuv/lib" LIBUV_CFLAGS="-I$/dependencies/libuv/include" CPPFLAGS=-I$/dependencies/libuv/include" LDFLAGS='-flat_namespace -force_flat_namespace' ./configure --prefix=/opt/target" --with-openssl="/dependencies/openssl" --disable-pthread-rwlock --with-libxml2=no --enable-full-report


There are two issues

1) make - FAILS

it fails because the declaration of the uv_handle_get_data and uv_handle_set_data does not match the declaration between uv-compat.h and  uv.h

In file included from netmgr/netmgr.c:33:
In file included from netmgr/netmgr-int.h:34:
netmgr/uv-compat.h:24:1: error: static declaration of 'uv_handle_get_data' follows non-static declaration
uv_handle_get_data(const uv_handle_t *handle) {
^
/opt/serverplus/dependencies/libuv/include/uv.h:448:17: note: previous declaration is here
UV_EXTERN void* uv_handle_get_data(const uv_handle_t* handle);
                ^
In file included from netmgr/netmgr.c:33:
In file included from netmgr/netmgr-int.h:34:
netmgr/uv-compat.h:31:1: error: static declaration of 'uv_handle_set_data' follows non-static declaration
uv_handle_set_data(uv_handle_t *handle, void *data) {
^
/opt/serverplus/dependencies/libuv/include/uv.h:450:16: note: previous declaration is here
UV_EXTERN void uv_handle_set_data(uv_handle_t* handle, void* data);
               ^
2 errors generated.
make[4]: *** [netmgr/libisc_la-netmgr.lo] Error 1
make[3]: *** [all-recursive] Error 1
make[2]: *** [all-recursive] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2


If I move the declaration from uv-compat.h to uv-compat.c then it compile and we removed the inlining.

Comment out these:

#ifndef HAVE_UV_HANDLE_GET_DATA
static inline void *
uv_handle_get_data(const uv_handle_t *handle) {
        return (handle->data);
}
#endif /* ifndef HAVE_UV_HANDLE_GET_DATA */

#ifndef HAVE_UV_HANDLE_SET_DATA
static inline void
uv_handle_set_data(uv_handle_t *handle, void *data) {
        handle->data = data;
}
#endif /* ifndef HAVE_UV_HANDLE_SET_DATA */

Add to uv-compat.c

#ifndef HAVE_UV_HANDLE_GET_DATA
UV_EXTERN void *
uv_handle_get_data(const uv_handle_t *handle) {
        return (handle->data);
}
#endif /* ifndef HAVE_UV_HANDLE_GET_DATA */

#ifndef HAVE_UV_HANDLE_SET_DATA
UV_EXTERN void
uv_handle_set_data(uv_handle_t *handle, void *data) {
        handle->data = data;
}
#endif /* ifndef HAVE_UV_HANDLE_SET_DATA */



2) deployed executable fails to load lib (deployed using make install)

Once the binaries are built and try to start the named up receive the following error

./named -c /named.conf -f -d4


dyld: lazy symbol binding failed: Symbol not found: _uv_loop_init
  Referenced from: <full path herer>lib/libisc.1701.dylib
  Expected in: flat namespace

dyld: Symbol not found: _uv_loop_init
  Referenced from: <full path here>lib/libisc.1701.dylib
  Expected in: flat namespace


So, it has something to do with flat namespaces vs. two level namespaces. I have tried to add the linker option per macOS documentation as LDFLAGS='-flat_namespace -force_flat_namespace but the executables and libraries still have to level namespaces

Tried to force lib to have flat level as well

otool -hV  <full path here>/lib/libuv.dylib                                             

Mach header
      magic cputype cpusubtype  caps    filetype ncmds sizeofcmds      flags
MH_MAGIC_64  X86_64        ALL  0x00       DYLIB    14       1640   NOUNDEFS DYLDLINK TWOLEVEL NO_REEXPORTED_DYLIBS


otool -hV <full path here>/lib/libisc.1701.dylib
Mach header
      magic cputype cpusubtype  caps    filetype ncmds sizeofcmds      flags
MH_MAGIC_64  X86_64        ALL  0x00       DYLIB    15       1952   NOUNDEFS DYLDLINK TWOLEVEL NO_REEXPORTED_DYLIBS MH_HAS_TLV_DESCRIPTORS


Can problem #1 fixed in source?
Does anyone have any idea how fix #2?


Thanks,


Eddy








_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/bind-users
Reply | Threaded
Open this post in threaded view
|

Re: Cannot build on macOS 10.15 (Catalina)

Ondřej Surý
Hi Eddy,

as I have already told you in the issue you’ve created, there error when building comes from the configure step. The way you mangle LIBUV_CFLAGS and LIBUV_LDFLAGS is wrong. You need to set correct PKG_CONFIG_PATH (to patch that contains libuv.pc), so configure finds libuv.

To fix the runtime problem you need to configure dynamic linker to find the libuv library. (Or use rpath linker option.)

Actually both problems stems from the fact that you installed libuv into nonstandard location. I would suggest to use homebrew or macports to install the dependencies.

Ondrej
--
Ondřej Surý — ISC

On 28 Apr 2020, at 19:37, Eddy Hahn <[hidden email]> wrote:

 Hi,

I got the latest source and build tools. 

1) autorecon -If. WORKED!

2) configure WORKED!

LIBUV_LIBS="-L$/dependencies/libuv/lib" LIBUV_CFLAGS="-I$/dependencies/libuv/include" CPPFLAGS=-I$/dependencies/libuv/include" LDFLAGS='-flat_namespace -force_flat_namespace' ./configure --prefix=/opt/target" --with-openssl="/dependencies/openssl" --disable-pthread-rwlock --with-libxml2=no --enable-full-report


There are two issues

1) make - FAILS

it fails because the declaration of the uv_handle_get_data and uv_handle_set_data does not match the declaration between uv-compat.h and  uv.h

In file included from netmgr/netmgr.c:33:
In file included from netmgr/netmgr-int.h:34:
netmgr/uv-compat.h:24:1: error: static declaration of 'uv_handle_get_data' follows non-static declaration
uv_handle_get_data(const uv_handle_t *handle) {
^
/opt/serverplus/dependencies/libuv/include/uv.h:448:17: note: previous declaration is here
UV_EXTERN void* uv_handle_get_data(const uv_handle_t* handle);
                ^
In file included from netmgr/netmgr.c:33:
In file included from netmgr/netmgr-int.h:34:
netmgr/uv-compat.h:31:1: error: static declaration of 'uv_handle_set_data' follows non-static declaration
uv_handle_set_data(uv_handle_t *handle, void *data) {
^
/opt/serverplus/dependencies/libuv/include/uv.h:450:16: note: previous declaration is here
UV_EXTERN void uv_handle_set_data(uv_handle_t* handle, void* data);
               ^
2 errors generated.
make[4]: *** [netmgr/libisc_la-netmgr.lo] Error 1
make[3]: *** [all-recursive] Error 1
make[2]: *** [all-recursive] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2


If I move the declaration from uv-compat.h to uv-compat.c then it compile and we removed the inlining.

Comment out these:

#ifndef HAVE_UV_HANDLE_GET_DATA
static inline void *
uv_handle_get_data(const uv_handle_t *handle) {
        return (handle->data);
}
#endif /* ifndef HAVE_UV_HANDLE_GET_DATA */

#ifndef HAVE_UV_HANDLE_SET_DATA
static inline void
uv_handle_set_data(uv_handle_t *handle, void *data) {
        handle->data = data;
}
#endif /* ifndef HAVE_UV_HANDLE_SET_DATA */

Add to uv-compat.c

#ifndef HAVE_UV_HANDLE_GET_DATA
UV_EXTERN void *
uv_handle_get_data(const uv_handle_t *handle) {
        return (handle->data);
}
#endif /* ifndef HAVE_UV_HANDLE_GET_DATA */

#ifndef HAVE_UV_HANDLE_SET_DATA
UV_EXTERN void
uv_handle_set_data(uv_handle_t *handle, void *data) {
        handle->data = data;
}
#endif /* ifndef HAVE_UV_HANDLE_SET_DATA */



2) deployed executable fails to load lib (deployed using make install)

Once the binaries are built and try to start the named up receive the following error

./named -c /named.conf -f -d4


dyld: lazy symbol binding failed: Symbol not found: _uv_loop_init
  Referenced from: <full path herer>lib/libisc.1701.dylib
  Expected in: flat namespace

dyld: Symbol not found: _uv_loop_init
  Referenced from: <full path here>lib/libisc.1701.dylib
  Expected in: flat namespace


So, it has something to do with flat namespaces vs. two level namespaces. I have tried to add the linker option per macOS documentation as LDFLAGS='-flat_namespace -force_flat_namespace but the executables and libraries still have to level namespaces

Tried to force lib to have flat level as well

otool -hV  <full path here>/lib/libuv.dylib                                             

Mach header
      magic cputype cpusubtype  caps    filetype ncmds sizeofcmds      flags
MH_MAGIC_64  X86_64        ALL  0x00       DYLIB    14       1640   NOUNDEFS DYLDLINK TWOLEVEL NO_REEXPORTED_DYLIBS


otool -hV <full path here>/lib/libisc.1701.dylib
Mach header
      magic cputype cpusubtype  caps    filetype ncmds sizeofcmds      flags
MH_MAGIC_64  X86_64        ALL  0x00       DYLIB    15       1952   NOUNDEFS DYLDLINK TWOLEVEL NO_REEXPORTED_DYLIBS MH_HAS_TLV_DESCRIPTORS


Can problem #1 fixed in source?
Does anyone have any idea how fix #2?


Thanks,


Eddy







_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/bind-users

_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/bind-users
Reply | Threaded
Open this post in threaded view
|

Re: Cannot build on macOS 10.15 (Catalina)

Tony Finch
In my experience getting rpaths to work properly is a massive pain because
most autoconf/libtool build systems don't automatically set the rpath as
required for the --with-libwhatever=PATH options to work properly, and
they often prevent attempts to set rpath linker flags. In BIND there has
been a fair amount of churn in this area so you should expect to need to
re-do any workarounds for each major version.

The main dependency that I build myself is OpenSSL (for Ed25519 support)
and my current build script sets the following environment variables
before running `./configure`.

export OPENSSL_CFLAGS="-I$OpenSSL/include"
export OPENSSL_LIBS="-L$OpenSSL/lib -Wl,-R,$OpenSSL/lib -lcrypto "

There seem to be similar LIBUV_ variables, so you might find they do the
trick for you. Alternatively, I install dnstap in the same $PREFIX as
BIND, and it's much less troublesome than OpenSSL, so you might find
that's also true for libuv.

Another option (to avoid fighting the build scripts) might be to use the
`chrpath` utility after building, but that appears to be specific to ELF.
There's a macho_edit which can alter rpaths, but building that in order to
build BIND might be too many yaks to shave...

(Building autoconfiscated code to link to libraries installed in /opt or
~/lib or wherever has been utterly horrible since the 1990s and libtool
has made it much slower and harder to debug, so I don't expect it will
ever work well.)

Tony.
--
f.anthony.n.finch  <[hidden email]>  http://dotat.at/
Plymouth: Northwest 3 to 5 backing south or southwest 5 to 7, perhaps gale 8
later. Slight or moderate, becoming rough later. Rain or showers. Good,
occasionally poor.
_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/bind-users
Reply | Threaded
Open this post in threaded view
|

Re: Cannot build on macOS 10.15 (Catalina)

Ondřej Surý
> On 28 Apr 2020, at 21:15, Tony Finch <[hidden email]> wrote:
>
> (Building autoconfiscated code to link to libraries installed in /opt or
> ~/lib or wherever has been utterly horrible since the 1990s and libtool
> has made it much slower and harder to debug, so I don't expect it will
> ever work well.)

On Linux, just put the path to /etc/ld.so.conf.d/local.conf and that should
do the trick. I don’t know how to configure the dynamic linker on macOS.

Ondrej
--
Ondřej Surý
[hidden email]



_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/bind-users

signature.asc (981 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Cannot build on macOS 10.15 (Catalina)

Ondřej Surý
In reply to this post by Eddy Hahn
LIBUV_LIBS="-L$/dependencies/libuv/lib“

JFTR this part of the line is wrong as it actually doesn’t contain the library itself (just LDFLAGS).
You should really use the pkgconfig.

Ondrej
--
Ondřej Surý
[hidden email]

> On 28 Apr 2020, at 19:36, Eddy Hahn <[hidden email]> wrote:
>
> LIBUV_LIBS="-L$/dependencies/libuv/lib"


_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/bind-users

signature.asc (981 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Cannot build on macOS 10.15 (Catalina)

PGNet Dev
In reply to this post by Ondřej Surý
On 4/28/20 12:21 PM, Ondřej Surý wrote:
> On Linux, just put the path to /etc/ld.so.conf.d/local.conf and that should
> do the trick. I don’t know how to configure the dynamic linker on macOS.

runtime dynamic linked paths, subject to ENV var changes, are, imo&e, simply a bad idea/recommendation. particularly for servers where you want the bins linked with the libs you want/specify, and never anything else.

fyi,

  https://tldp.org/HOWTO/Program-Library-HOWTO/shared-libraries.html

 "...
  LD_LIBRARY_PATH is handy for development and testing, but shouldn't be modified by an installation process for normal use by normal users;
  ..."

and,

  http://xahlee.info/UnixResource_dir/_/ldpath.html


fwiw, rpath'ing with bind, usually in LDFLAGS to avoid sometimes/historically flaky collisions with the lib path assumptions in autofoo & pkg-config config , has worked fine here for quite awhile; e.g., for openssl, lmdb, geoip & libuv.

$0.02 & ymmv.
_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/bind-users
Reply | Threaded
Open this post in threaded view
|

Re: Cannot build on macOS 10.15 (Catalina)

Eddy Hahn
In reply to this post by Ondřej Surý

OK. Before I did not give you the full picture because I did not want to be to verbose :-)

It should have been

export SERVERPLUS_DIR="/opt/serverplus”
cd bind9
autoreconf -if
LIBUV_LIBS="-L$SERVERPLUS_DIR/dependencies/libuv/lib" LIBUV_CFLAGS=" -I$SERVERPLUS_DIR/dependencies/libuv/include" CPPFLAGS=" -I$SERVERPLUS_DIR/dependencies/libuv/include" LDFLAGS='-flat_namespace -force_flat_namespace' ./configure --prefix="$SERVERPLUS_DIR" --with-openssl="$SERVERPLUS_DIR/dependencies/openssl" --disable-pthread-rwlock --with-libxml2=no --enable-full-report



After Ondrej suggestion I have removed all the LIBUV_XXX env variables and change to

PGK_CONFIG_PATH="/opt/build/tools/pkg-config" PKG_CONFIG_PATH="$SERVERPLUS_DIR/dependencies/libuv/lib/pkgconfig" ./configure --prefix="$SERVERPLUS_DIR" --with-openssl="$SERVERPLUS_DIR/dependencies/openssl" --disable-pthread-rwlock --with-libxml2=no --enable-full-report


I did not had to change the source code :-) and after deploying no more complaints from dylib! named is running fine!

All worked! Thank you very much for the help!

Thanks again,

Eddy



On Apr 28, 2020, at 12:22 PM, Ondřej Surý <[hidden email]> wrote:

LIBUV_LIBS="-L$/dependencies/libuv/lib“

JFTR this part of the line is wrong as it actually doesn’t contain the library itself (just LDFLAGS).
You should really use the pkgconfig.

Ondrej
--
Ondřej Surý
[hidden email]

On 28 Apr 2020, at 19:36, Eddy Hahn <[hidden email]> wrote:

LIBUV_LIBS="-L$/dependencies/libuv/lib"



_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/bind-users
Reply | Threaded
Open this post in threaded view
|

Re: Cannot build on macOS 10.15 (Catalina)

Ondřej Surý
PGK_CONFIG_PATH="/opt/build/tools/pkg-config"

^^^ there’s a typo and if you need to add paths to more libraries, you need to use colon, like this:

PKG_CONFIG_PATH=/usr/local/opt/libxml2/lib/pkgconfig:/usr/local/opt/zlib/lib/pkgconfig:

(taken from my work machine with homebrew versions of libxml2 and zlib)

Ondrej
--
Ondřej Surý
[hidden email]

> On 28 Apr 2020, at 22:12, Eddy Hahn <[hidden email]> wrote:
>
>
> OK. Before I did not give you the full picture because I did not want to be to verbose :-)
>
> It should have been
>
> export SERVERPLUS_DIR="/opt/serverplus”
> git clone https://gitlab.isc.org/isc-projects/bind9.git
> cd bind9
> autoreconf -if
> LIBUV_LIBS="-L$SERVERPLUS_DIR/dependencies/libuv/lib" LIBUV_CFLAGS=" -I$SERVERPLUS_DIR/dependencies/libuv/include" CPPFLAGS=" -I$SERVERPLUS_DIR/dependencies/libuv/include" LDFLAGS='-flat_namespace -force_flat_namespace' ./configure --prefix="$SERVERPLUS_DIR" --with-openssl="$SERVERPLUS_DIR/dependencies/openssl" --disable-pthread-rwlock --with-libxml2=no --enable-full-report
>
>
>
> After Ondrej suggestion I have removed all the LIBUV_XXX env variables and change to
>
> PGK_CONFIG_PATH="/opt/build/tools/pkg-config" PKG_CONFIG_PATH="$SERVERPLUS_DIR/dependencies/libuv/lib/pkgconfig" ./configure --prefix="$SERVERPLUS_DIR" --with-openssl="$SERVERPLUS_DIR/dependencies/openssl" --disable-pthread-rwlock --with-libxml2=no --enable-full-report
>
>
> I did not had to change the source code :-) and after deploying no more complaints from dylib! named is running fine!
>
> All worked! Thank you very much for the help!
>
> Thanks again,
>
> Eddy
>
>
>
>> On Apr 28, 2020, at 12:22 PM, Ondřej Surý <[hidden email]> wrote:
>>
>> LIBUV_LIBS="-L$/dependencies/libuv/lib“
>>
>> JFTR this part of the line is wrong as it actually doesn’t contain the library itself (just LDFLAGS).
>> You should really use the pkgconfig.
>>
>> Ondrej
>> --
>> Ondřej Surý
>> [hidden email]
>>
>>> On 28 Apr 2020, at 19:36, Eddy Hahn <[hidden email]> wrote:
>>>
>>> LIBUV_LIBS="-L$/dependencies/libuv/lib"
>>
>

_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/bind-users

signature.asc (981 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Cannot build on macOS 10.15 (Catalina)

Tony Finch
In reply to this post by Ondřej Surý
Ondřej Surý <[hidden email]> wrote:
>
> On Linux, just put the path to /etc/ld.so.conf.d/local.conf and that should
> do the trick.

I'm usually using per-build install paths for experimentation or for easy
rollback, so I prefer not to fiddle with the global path. I make things
difficult for myself :-)

Tony.
--
f.anthony.n.finch  <[hidden email]>  http://dotat.at/
Thames, Dover: Cyclonic becoming south 5 or 6. Slight or moderate. Rain or
showers. Good, occasionally moderate.
_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/bind-users