Bind 9.11.13 - inline re-signing stops

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

Bind 9.11.13 - inline re-signing stops

Matthew Richardson
I have an interesting issue with a hidden master running 9.11.13 and
configured with inline signing on a number of zones, configured thus:-

>zone "42.201.193.in-addr.arpa" {
>    type master;
>    file "zones/master/42.201.193.in-addr.arpa.db";
>    inline-signing yes;
>    auto-dnssec maintain;
>};

Prior to 30 January, all the zones configured in this way were regurlarly
being resigned, logging entries such as:-

>29-Jan-2020 03:37:02.129 general: info: zone 42.201.193.in-addr.arpa/IN (signed): reconfiguring zone keys
>29-Jan-2020 03:37:02.131 general: info: zone 42.201.193.in-addr.arpa/IN (signed): next key event: 29-Jan-2020 15:37:02.129
>29-Jan-2020 15:37:02.129 general: info: zone 42.201.193.in-addr.arpa/IN (signed): reconfiguring zone keys
>29-Jan-2020 15:37:02.131 general: info: zone 42.201.193.in-addr.arpa/IN (signed): next key event: 30-Jan-2020 03:37:02.129
>30-Jan-2020 03:35:01.604 general: info: zone 42.201.193.in-addr.arpa/IN (signed): reconfiguring zone keys
>30-Jan-2020 03:35:01.606 general: info: zone 42.201.193.in-addr.arpa/IN (signed): next key event: 30-Jan-2020 15:35:01.604

Since an "rndc reload" at 12:22 on 30 January, this logging has stopped and
NONE of the signed zones have had any of their RRSIGs re-signed.  Today,
one sees:-

>[root@m70 dns]# rndc zonestatus 42.201.193.in-addr.arpa
>name: 42.201.193.in-addr.arpa
>type: master
>files: zones/master/42.201.193.in-addr.arpa.db
>serial: 286
>signed serial: 3829
>nodes: 140
>last loaded: Sun, 24 Nov 2019 07:13:00 GMT
>secure: yes
>inline signing: yes
>key maintenance: automatic
>next key event: Wed, 05 Feb 2020 18:01:42 GMT
>next resign node: DI2VMBB2GDES2IKFVFRUB7DIDDC7TI8L.42.201.193.in-addr.arpa/NSEC3
>next resign time: Thu, 30 Jan 2020 21:25:35 GMT
>dynamic: no
>reconfigurable via modzone: no

which clearly shows "next resign" as being in the past.  The server
reports:-

>[root@m70 dns]# rndc status
>version: BIND 9.11.13 (Extended Support Version) <id:ad4df16>
>running on m70: Linux x86_64 4.14.120-x86_64-linode125 #1 SMP Mon May 20 16:43:35 UTC 2019
>boot time: Sun, 24 Nov 2019 09:51:27 GMT
>last configured: Wed, 05 Feb 2020 18:10:21 GMT
>configuration file: /etc/named.conf
>CPUs found: 1
>worker threads: 1
>UDP listeners per interface: 1
>number of zones: 773 (0 automatic)
>debug level: 0
>xfers running: 0
>xfers deferred: 0
>soa queries in progress: 0
>query logging is OFF
>recursive clients: 0/900/1000
>tcp clients: 6/150
>TCP high-water: 64
>server is up and running

As a test I tried incrementing the serial number of only one of the signed
zones and, after a reload, that zone seems to be being resigned normally.

My suspicion is that retarting Bind will simply fix the issue.

However, I was wondering whether there might be any troubleshooting or
diagnosis which it might be useful to undertake.  Were ISC to want, it
would probably be possible to get them temporary access.

Best wishes,
Matthew
_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/bind-users
Reply | Threaded
Open this post in threaded view
|

Re: Bind 9.11.13 - inline re-signing stops

Ondřej Surý
Hi Matthew,

we have fixed a data race in a related code in BIND 9.11.15.

However, if you can get us a coredump using gdb and tar it with associated binary and libraries, we might be able to look into it. ISC GitLab should have enough limit to accept tar.xz, make sure you set the issue as confidential, we will sanitize it before making the issue public in the future. You may use pandora.isc.org to drop of larger files in a confidential matter and link it to the GitLab issue.

Ondřej
--
Ondřej Surý — ISC

> On 5 Feb 2020, at 19:28, Matthew Richardson <[hidden email]> wrote:
>
> I have an interesting issue with a hidden master running 9.11.13 and
> configured with inline signing on a number of zones, configured thus:-
>
>> zone "42.201.193.in-addr.arpa" {
>>   type master;
>>   file "zones/master/42.201.193.in-addr.arpa.db";
>>   inline-signing yes;
>>   auto-dnssec maintain;
>> };
>
> Prior to 30 January, all the zones configured in this way were regurlarly
> being resigned, logging entries such as:-
>
>> 29-Jan-2020 03:37:02.129 general: info: zone 42.201.193.in-addr.arpa/IN (signed): reconfiguring zone keys
>> 29-Jan-2020 03:37:02.131 general: info: zone 42.201.193.in-addr.arpa/IN (signed): next key event: 29-Jan-2020 15:37:02.129
>> 29-Jan-2020 15:37:02.129 general: info: zone 42.201.193.in-addr.arpa/IN (signed): reconfiguring zone keys
>> 29-Jan-2020 15:37:02.131 general: info: zone 42.201.193.in-addr.arpa/IN (signed): next key event: 30-Jan-2020 03:37:02.129
>> 30-Jan-2020 03:35:01.604 general: info: zone 42.201.193.in-addr.arpa/IN (signed): reconfiguring zone keys
>> 30-Jan-2020 03:35:01.606 general: info: zone 42.201.193.in-addr.arpa/IN (signed): next key event: 30-Jan-2020 15:35:01.604
>
> Since an "rndc reload" at 12:22 on 30 January, this logging has stopped and
> NONE of the signed zones have had any of their RRSIGs re-signed.  Today,
> one sees:-
>
>> [root@m70 dns]# rndc zonestatus 42.201.193.in-addr.arpa
>> name: 42.201.193.in-addr.arpa
>> type: master
>> files: zones/master/42.201.193.in-addr.arpa.db
>> serial: 286
>> signed serial: 3829
>> nodes: 140
>> last loaded: Sun, 24 Nov 2019 07:13:00 GMT
>> secure: yes
>> inline signing: yes
>> key maintenance: automatic
>> next key event: Wed, 05 Feb 2020 18:01:42 GMT
>> next resign node: DI2VMBB2GDES2IKFVFRUB7DIDDC7TI8L.42.201.193.in-addr.arpa/NSEC3
>> next resign time: Thu, 30 Jan 2020 21:25:35 GMT
>> dynamic: no
>> reconfigurable via modzone: no
>
> which clearly shows "next resign" as being in the past.  The server
> reports:-
>
>> [root@m70 dns]# rndc status
>> version: BIND 9.11.13 (Extended Support Version) <id:ad4df16>
>> running on m70: Linux x86_64 4.14.120-x86_64-linode125 #1 SMP Mon May 20 16:43:35 UTC 2019
>> boot time: Sun, 24 Nov 2019 09:51:27 GMT
>> last configured: Wed, 05 Feb 2020 18:10:21 GMT
>> configuration file: /etc/named.conf
>> CPUs found: 1
>> worker threads: 1
>> UDP listeners per interface: 1
>> number of zones: 773 (0 automatic)
>> debug level: 0
>> xfers running: 0
>> xfers deferred: 0
>> soa queries in progress: 0
>> query logging is OFF
>> recursive clients: 0/900/1000
>> tcp clients: 6/150
>> TCP high-water: 64
>> server is up and running
>
> As a test I tried incrementing the serial number of only one of the signed
> zones and, after a reload, that zone seems to be being resigned normally.
>
> My suspicion is that retarting Bind will simply fix the issue.
>
> However, I was wondering whether there might be any troubleshooting or
> diagnosis which it might be useful to undertake.  Were ISC to want, it
> would probably be possible to get them temporary access.
>
> Best wishes,
> Matthew
> _______________________________________________
> Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list
>
> bind-users mailing list
> [hidden email]
> https://lists.isc.org/mailman/listinfo/bind-users

_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/bind-users
Reply | Threaded
Open this post in threaded view
|

VS: Bind 9.11.13 - inline re-signing stops

Jukka Pakkanen
In reply to this post by Matthew Richardson
Maybe related to your earlier reported problem, when signed zonedata is not updated after updates to the zone?

And what I already read about 9.11.15, hopefully fixed there.

Jukka

-----Alkuperäinen viesti-----
Lähettäjä: bind-users <[hidden email]> Puolesta Matthew Richardson
Lähetetty: 5. helmikuuta 2020 19:28
Vastaanottaja: [hidden email]
Aihe: Bind 9.11.13 - inline re-signing stops

I have an interesting issue with a hidden master running 9.11.13 and configured with inline signing on a number of zones, configured thus:-

>zone "42.201.193.in-addr.arpa" {
>    type master;
>    file "zones/master/42.201.193.in-addr.arpa.db";
>    inline-signing yes;
>    auto-dnssec maintain;
>};

Prior to 30 January, all the zones configured in this way were regurlarly being resigned, logging entries such as:-

>29-Jan-2020 03:37:02.129 general: info: zone 42.201.193.in-addr.arpa/IN
>(signed): reconfiguring zone keys
>29-Jan-2020 03:37:02.131 general: info: zone 42.201.193.in-addr.arpa/IN
>(signed): next key event: 29-Jan-2020 15:37:02.129
>29-Jan-2020 15:37:02.129 general: info: zone 42.201.193.in-addr.arpa/IN
>(signed): reconfiguring zone keys
>29-Jan-2020 15:37:02.131 general: info: zone 42.201.193.in-addr.arpa/IN
>(signed): next key event: 30-Jan-2020 03:37:02.129
>30-Jan-2020 03:35:01.604 general: info: zone 42.201.193.in-addr.arpa/IN
>(signed): reconfiguring zone keys
>30-Jan-2020 03:35:01.606 general: info: zone 42.201.193.in-addr.arpa/IN
>(signed): next key event: 30-Jan-2020 15:35:01.604

Since an "rndc reload" at 12:22 on 30 January, this logging has stopped and NONE of the signed zones have had any of their RRSIGs re-signed.  Today, one sees:-

>[root@m70 dns]# rndc zonestatus 42.201.193.in-addr.arpa
>name: 42.201.193.in-addr.arpa
>type: master
>files: zones/master/42.201.193.in-addr.arpa.db
>serial: 286
>signed serial: 3829
>nodes: 140
>last loaded: Sun, 24 Nov 2019 07:13:00 GMT
>secure: yes
>inline signing: yes
>key maintenance: automatic
>next key event: Wed, 05 Feb 2020 18:01:42 GMT next resign node:
>DI2VMBB2GDES2IKFVFRUB7DIDDC7TI8L.42.201.193.in-addr.arpa/NSEC3
>next resign time: Thu, 30 Jan 2020 21:25:35 GMT
>dynamic: no
>reconfigurable via modzone: no

which clearly shows "next resign" as being in the past.  The server
reports:-

>[root@m70 dns]# rndc status
>version: BIND 9.11.13 (Extended Support Version) <id:ad4df16> running
>on m70: Linux x86_64 4.14.120-x86_64-linode125 #1 SMP Mon May 20
>16:43:35 UTC 2019 boot time: Sun, 24 Nov 2019 09:51:27 GMT last
>configured: Wed, 05 Feb 2020 18:10:21 GMT configuration file:
>/etc/named.conf CPUs found: 1 worker threads: 1 UDP listeners per
>interface: 1 number of zones: 773 (0 automatic) debug level: 0 xfers
>running: 0 xfers deferred: 0 soa queries in progress: 0 query logging
>is OFF recursive clients: 0/900/1000 tcp clients: 6/150 TCP high-water:
>64 server is up and running

As a test I tried incrementing the serial number of only one of the signed zones and, after a reload, that zone seems to be being resigned normally.

My suspicion is that retarting Bind will simply fix the issue.

However, I was wondering whether there might be any troubleshooting or diagnosis which it might be useful to undertake.  Were ISC to want, it would probably be possible to get them temporary access.

Best wishes,
Matthew
_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/bind-users

_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/bind-users
Reply | Threaded
Open this post in threaded view
|

VS: Bind 9.11.13 - inline re-signing stops

Jukka Pakkanen
..except our problem is with the 9.14.9 version/branch.


-----Alkuperäinen viesti-----
Lähettäjä: Jukka Pakkanen
Lähetetty: 5. helmikuuta 2020 23:52
Vastaanottaja: 'Matthew Richardson' <[hidden email]>; [hidden email]
Aihe: VS: Bind 9.11.13 - inline re-signing stops

Maybe related to your earlier reported problem, when signed zonedata is not updated after updates to the zone?

And what I already read about 9.11.15, hopefully fixed there.

Jukka

-----Alkuperäinen viesti-----
Lähettäjä: bind-users <[hidden email]> Puolesta Matthew Richardson
Lähetetty: 5. helmikuuta 2020 19:28
Vastaanottaja: [hidden email]
Aihe: Bind 9.11.13 - inline re-signing stops

I have an interesting issue with a hidden master running 9.11.13 and configured with inline signing on a number of zones, configured thus:-

>zone "42.201.193.in-addr.arpa" {
>    type master;
>    file "zones/master/42.201.193.in-addr.arpa.db";
>    inline-signing yes;
>    auto-dnssec maintain;
>};

Prior to 30 January, all the zones configured in this way were regurlarly being resigned, logging entries such as:-

>29-Jan-2020 03:37:02.129 general: info: zone 42.201.193.in-addr.arpa/IN
>(signed): reconfiguring zone keys
>29-Jan-2020 03:37:02.131 general: info: zone 42.201.193.in-addr.arpa/IN
>(signed): next key event: 29-Jan-2020 15:37:02.129
>29-Jan-2020 15:37:02.129 general: info: zone 42.201.193.in-addr.arpa/IN
>(signed): reconfiguring zone keys
>29-Jan-2020 15:37:02.131 general: info: zone 42.201.193.in-addr.arpa/IN
>(signed): next key event: 30-Jan-2020 03:37:02.129
>30-Jan-2020 03:35:01.604 general: info: zone 42.201.193.in-addr.arpa/IN
>(signed): reconfiguring zone keys
>30-Jan-2020 03:35:01.606 general: info: zone 42.201.193.in-addr.arpa/IN
>(signed): next key event: 30-Jan-2020 15:35:01.604

Since an "rndc reload" at 12:22 on 30 January, this logging has stopped and NONE of the signed zones have had any of their RRSIGs re-signed.  Today, one sees:-

>[root@m70 dns]# rndc zonestatus 42.201.193.in-addr.arpa
>name: 42.201.193.in-addr.arpa
>type: master
>files: zones/master/42.201.193.in-addr.arpa.db
>serial: 286
>signed serial: 3829
>nodes: 140
>last loaded: Sun, 24 Nov 2019 07:13:00 GMT
>secure: yes
>inline signing: yes
>key maintenance: automatic
>next key event: Wed, 05 Feb 2020 18:01:42 GMT next resign node:
>DI2VMBB2GDES2IKFVFRUB7DIDDC7TI8L.42.201.193.in-addr.arpa/NSEC3
>next resign time: Thu, 30 Jan 2020 21:25:35 GMT
>dynamic: no
>reconfigurable via modzone: no

which clearly shows "next resign" as being in the past.  The server
reports:-

>[root@m70 dns]# rndc status
>version: BIND 9.11.13 (Extended Support Version) <id:ad4df16> running
>on m70: Linux x86_64 4.14.120-x86_64-linode125 #1 SMP Mon May 20
>16:43:35 UTC 2019 boot time: Sun, 24 Nov 2019 09:51:27 GMT last
>configured: Wed, 05 Feb 2020 18:10:21 GMT configuration file:
>/etc/named.conf CPUs found: 1 worker threads: 1 UDP listeners per
>interface: 1 number of zones: 773 (0 automatic) debug level: 0 xfers
>running: 0 xfers deferred: 0 soa queries in progress: 0 query logging
>is OFF recursive clients: 0/900/1000 tcp clients: 6/150 TCP high-water:
>64 server is up and running

As a test I tried incrementing the serial number of only one of the signed zones and, after a reload, that zone seems to be being resigned normally.

My suspicion is that retarting Bind will simply fix the issue.

However, I was wondering whether there might be any troubleshooting or diagnosis which it might be useful to undertake.  Were ISC to want, it would probably be possible to get them temporary access.

Best wishes,
Matthew
_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/bind-users

_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/bind-users
Reply | Threaded
Open this post in threaded view
|

Re: Bind 9.11.13 - inline re-signing stops

Matthew Richardson
In reply to this post by Ondřej Surý
Having upgraded to 9.11.15 I am still seeing similar problems, where some
zones stop updating their signatures.  I have a suspicion that "rndc
reconfig" might get them re-signing, or perhaps updating the serial of the
unsigned zone.

I have logged it on GitLab as #1627, and will take your advice of marking
it private as I will be uploading config, zonefiles and logs.  If the
attachments are removed, I am happy for it to be public as they are the
only confidential part.

Best wishes,
Matthew
 ------
>From: Ond?ej Surý <[hidden email]>
>To: Matthew Richardson <[hidden email]>
>Cc: [hidden email]
>Date: Wed, 5 Feb 2020 20:02:17 +0100
>Subject: Re: Bind 9.11.13 - inline re-signing stops

>Hi Matthew,
>
>we have fixed a data race in a related code in BIND 9.11.15.
>
>However, if you can get us a coredump using gdb and tar it with associated binary and libraries, we might be able to look into it. ISC GitLab should have enough limit to accept tar.xz, make sure you set the issue as confidential, we will sanitize it before making the issue public in the future. You may use pandora.isc.org to drop of larger files in a confidential matter and link it to the GitLab issue.
>
>Ond?ej

_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/bind-users
Reply | Threaded
Open this post in threaded view
|

Re: Bind 9.11.13 - inline re-signing stops

Ondřej Surý
Matthew,

thank you for the report. Indeed, we have received some more reports about this particular issue and we are currently investigating it.

We would appreciate if anybody who encounters the problem could dump the core first before restarting the named, so we have more evidence to analyze.

Ondrej
--
Ondřej Surý — ISC

> On 18 Feb 2020, at 23:22, Matthew Richardson <[hidden email]> wrote:
>
> Having upgraded to 9.11.15 I am still seeing similar problems, where some
> zones stop updating their signatures.  I have a suspicion that "rndc
> reconfig" might get them re-signing, or perhaps updating the serial of the
> unsigned zone.
>
> I have logged it on GitLab as #1627, and will take your advice of marking
> it private as I will be uploading config, zonefiles and logs.  If the
> attachments are removed, I am happy for it to be public as they are the
> only confidential part.
>
> Best wishes,
> Matthew
> ------
>> From: Ond?ej Surý <[hidden email]>
>> To: Matthew Richardson <[hidden email]>
>> Cc: [hidden email]
>> Date: Wed, 5 Feb 2020 20:02:17 +0100
>> Subject: Re: Bind 9.11.13 - inline re-signing stops
>
>> Hi Matthew,
>>
>> we have fixed a data race in a related code in BIND 9.11.15.
>>
>> However, if you can get us a coredump using gdb and tar it with associated binary and libraries, we might be able to look into it. ISC GitLab should have enough limit to accept tar.xz, make sure you set the issue as confidential, we will sanitize it before making the issue public in the future. You may use pandora.isc.org to drop of larger files in a confidential matter and link it to the GitLab issue.
>>
>> Ond?ej
>
> _______________________________________________
> Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list
>
> bind-users mailing list
> [hidden email]
> https://lists.isc.org/mailman/listinfo/bind-users

_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/bind-users
Reply | Threaded
Open this post in threaded view
|

Re: Bind 9.11.13 - inline re-signing stops

Matthew Richardson
Dear Ondrej,

I would be delighted to assist with a core dump.

However, not being a developer myself (I did do an amount of assembler work
in my youth, but that was long ago now!), I therefore do not know how to
get you "a coredump using gdb and tar it with associated binary and
libraries".

Do you have any step by step instructions for a novice to achieve this?
The machine is running Centos 7, and Bind is built using rpmbuild from the
source rpm provided by ISC.

With many thanks.

Best wishes,
Matthew
 ------
>From: Ond?ej Surý <[hidden email]>
>To: Matthew Richardson <[hidden email]>
>Cc: [hidden email]
>Date: Wed, 19 Feb 2020 07:16:49 +0100
>Subject: Re: Bind 9.11.13 - inline re-signing stops

>Matthew,
>
>thank you for the report. Indeed, we have received some more reports about this particular issue and we are currently investigating it.
>
>We would appreciate if anybody who encounters the problem could dump the core first before restarting the named, so we have more evidence to analyze.
>
>Ondrej

_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/bind-users
Reply | Threaded
Open this post in threaded view
|

VS: Bind 9.11.13 - inline re-signing stops

Jukka Pakkanen
In reply to this post by Ondřej Surý
Like reported earlier, this same behavior/problem occurs also in 9.14.x branch, hopefully the fix will be found quickly, it is quite disturbing problem.

Jukka

-----Alkuperäinen viesti-----
Lähettäjä: bind-users <[hidden email]> Puolesta Ondrej Surý
Lähetetty: 19. helmikuuta 2020 7:17
Vastaanottaja: Matthew Richardson <[hidden email]>
Kopio: [hidden email]
Aihe: Re: Bind 9.11.13 - inline re-signing stops

Matthew,

thank you for the report. Indeed, we have received some more reports about this particular issue and we are currently investigating it.

We would appreciate if anybody who encounters the problem could dump the core first before restarting the named, so we have more evidence to analyze.

Ondrej
--
Ondřej Surý — ISC

> On 18 Feb 2020, at 23:22, Matthew Richardson <[hidden email]> wrote:
>
> Having upgraded to 9.11.15 I am still seeing similar problems, where
> some zones stop updating their signatures.  I have a suspicion that
> "rndc reconfig" might get them re-signing, or perhaps updating the
> serial of the unsigned zone.
>
> I have logged it on GitLab as #1627, and will take your advice of
> marking it private as I will be uploading config, zonefiles and logs.  
> If the attachments are removed, I am happy for it to be public as they
> are the only confidential part.
>
> Best wishes,
> Matthew
> ------
>> From: Ond?ej Surý <[hidden email]>
>> To: Matthew Richardson <[hidden email]>
>> Cc: [hidden email]
>> Date: Wed, 5 Feb 2020 20:02:17 +0100
>> Subject: Re: Bind 9.11.13 - inline re-signing stops
>
>> Hi Matthew,
>>
>> we have fixed a data race in a related code in BIND 9.11.15.
>>
>> However, if you can get us a coredump using gdb and tar it with associated binary and libraries, we might be able to look into it. ISC GitLab should have enough limit to accept tar.xz, make sure you set the issue as confidential, we will sanitize it before making the issue public in the future. You may use pandora.isc.org to drop of larger files in a confidential matter and link it to the GitLab issue.
>>
>> Ond?ej
>
> _______________________________________________
> Please visit https://lists.isc.org/mailman/listinfo/bind-users to
> unsubscribe from this list
>
> bind-users mailing list
> [hidden email]
> https://lists.isc.org/mailman/listinfo/bind-users

_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/bind-users
_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/bind-users
Reply | Threaded
Open this post in threaded view
|

Re: Bind 9.11.13 - inline re-signing stops

Tony Finch
In reply to this post by Matthew Richardson
Matthew Richardson <[hidden email]> wrote:

> Having upgraded to 9.11.15 I am still seeing similar problems, where some
> zones stop updating their signatures.

I recently had a signing problem on my toy server which I think was
caused by a cockup with `rndc freeze`. It was not easy to get named to
re-start re-signing the zones properly :-(

One symptom was that the broken zones had "resign" times in the past. I'm
using raw format zones without inline signing, so I can look at this with:

        named-compilezone -j -f raw -o /dev/stdout $zone $file |
        grep resign | sort -r

With inline-signing you want to look at the .signed file.

I tried deliberately breaking a zone with `rndc freeze` but it recovered
OK. One difference was that my deliberately broken zone had the same time
on all its signatures, so there wasn't a mixture of past and future resign
times.

So, no bright ideas here I'm afraid.

Tony.
--
f.anthony.n.finch  <[hidden email]>  http://dotat.at/
South Biscay, Southeast Fitzroy: Variable 2 to 4 becoming southwesterly 4 to
6. Rough or very rough. Rain later. Moderate or good.
_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/bind-users
Reply | Threaded
Open this post in threaded view
|

Re: Bind 9.11.13 - inline re-signing stops

Ondřej Surý
In reply to this post by Matthew Richardson
Hi Matthew,

we’ll write some generic instruction on how to get a coredump from a running named
(without crashing it) - generally you want to use gcore[1].

Then here you can continue as if the named has crashed and get us a thread stack trace[2].

As we need to access very specific data structure, please add the information to the issue
you have opened, and we’ll pick it up from there.  You can also share the coredump and the
binaries with us using pandora.isc.org service, but please be aware that the memory dump
can (and probably will) contain the DNSSEC signing private keys.

Ondrej

1. https://www.systutorials.com/docs/linux/man/1-gcore/
2. https://kb.isc.org/docs/aa-00340

Ondrej
--
Ondřej Surý
[hidden email]

> On 19 Feb 2020, at 08:36, Matthew Richardson <[hidden email]> wrote:
>
> Dear Ondrej,
>
> I would be delighted to assist with a core dump.
>
> However, not being a developer myself (I did do an amount of assembler work
> in my youth, but that was long ago now!), I therefore do not know how to
> get you "a coredump using gdb and tar it with associated binary and
> libraries".
>
> Do you have any step by step instructions for a novice to achieve this?
> The machine is running Centos 7, and Bind is built using rpmbuild from the
> source rpm provided by ISC.
>
> With many thanks.
>
> Best wishes,
> Matthew
> ------
>> From: Ond?ej Surý <[hidden email]>
>> To: Matthew Richardson <[hidden email]>
>> Cc: [hidden email]
>> Date: Wed, 19 Feb 2020 07:16:49 +0100
>> Subject: Re: Bind 9.11.13 - inline re-signing stops
>
>> Matthew,
>>
>> thank you for the report. Indeed, we have received some more reports about this particular issue and we are currently investigating it.
>>
>> We would appreciate if anybody who encounters the problem could dump the core first before restarting the named, so we have more evidence to analyze.
>>
>> Ondrej
>
> _______________________________________________
> Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list
>
> bind-users mailing list
> [hidden email]
> https://lists.isc.org/mailman/listinfo/bind-users

_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/bind-users
Reply | Threaded
Open this post in threaded view
|

Re: Bind 9.11.13 - inline re-signing stops

Matthew Richardson
Dear Ondrej,

Thank you for your advice below.  I have attempted a dump on the live
running server and have uploaded the results to issue #1627.

If I need to try again, please let me know...  :-)

There are a few more days before I need to restart Named.

Best wishes,
Matthew

 ------
>From: Ond?ej Surý <[hidden email]>
>To: Matthew Richardson <[hidden email]>
>Cc: BIND Users <[hidden email]>
>Date: Thu, 20 Feb 2020 18:07:50 +0100
>Subject: Re: Bind 9.11.13 - inline re-signing stops

>Hi Matthew,
>
>we’ll write some generic instruction on how to get a coredump from a running named
>(without crashing it) - generally you want to use gcore[1].
>
>Then here you can continue as if the named has crashed and get us a thread stack trace[2].
>
>As we need to access very specific data structure, please add the information to the issue
>you have opened, and we’ll pick it up from there.  You can also share the coredump and the
>binaries with us using pandora.isc.org service, but please be aware that the memory dump
>can (and probably will) contain the DNSSEC signing private keys.
>
>Ondrej
>
>1. https://www.systutorials.com/docs/linux/man/1-gcore/
>2. https://kb.isc.org/docs/aa-00340
>
>Ondrej

_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/bind-users
Reply | Threaded
Open this post in threaded view
|

Re: Bind 9.11.13 - inline re-signing stops

Matthew Richardson
Firstly a big thank you to Mark and Ondrej for their assistance, which
tracked down the issue.  I understand will be fixed in the next releases.

My particular issue seemed to relate to the unsigned zonefiles being
touched (by my automation) without the contents changing, followed by an
"rndc reload".  This caused some domains to stop re-signing, the symptom of
which could be seen by the lack of "next key event:" in the logs for the
failing domains.

It turned out that "rndc reconfig" fixed the issue, making it easy to work
around the problem.

Best wishes,
Matthew

 ------
>From: Matthew Richardson <[hidden email]>
>To: BIND Users <[hidden email]>
>Cc:
>Date: Sat, 22 Feb 2020 19:49:22 +0000
>Subject: Re: Bind 9.11.13 - inline re-signing stops

>Dear Ondrej,
>
>Thank you for your advice below.  I have attempted a dump on the live
>running server and have uploaded the results to issue #1627.
>
>If I need to try again, please let me know...  :-)
>
>There are a few more days before I need to restart Named.
>
>Best wishes,
>Matthew
>
> ------
>>From: Ond?ej Surý <[hidden email]>
>>To: Matthew Richardson <[hidden email]>
>>Cc: BIND Users <[hidden email]>
>>Date: Thu, 20 Feb 2020 18:07:50 +0100
>>Subject: Re: Bind 9.11.13 - inline re-signing stops
>
>>Hi Matthew,
>>
>>we’ll write some generic instruction on how to get a coredump from a running named
>>(without crashing it) - generally you want to use gcore[1].
>>
>>Then here you can continue as if the named has crashed and get us a thread stack trace[2].
>>
>>As we need to access very specific data structure, please add the information to the issue
>>you have opened, and we’ll pick it up from there.  You can also share the coredump and the
>>binaries with us using pandora.isc.org service, but please be aware that the memory dump
>>can (and probably will) contain the DNSSEC signing private keys.
>>
>>Ondrej
>>
>>1. https://www.systutorials.com/docs/linux/man/1-gcore/
>>2. https://kb.isc.org/docs/aa-00340
>>
>>Ondrej

_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/bind-users