[Community-Discuss] 06 April 2019 RPKI incident - Postmortem report

[Cedrick Adrien Mbeyet]

Dear AFRINIC community,


Find below postmortem report on the incident that happen on 06 April 2019.

 

The AFRINIC RPKI engine has an offline part that has to be renewed on a
monthly bases. The process is known, documented and automated reminders
set. The system is set to send 2 reminders each month, one 15 days prior
to the expiry date and the second one 7 days before expiry. On the 2nd
half of March, the monitoring system sent a reminder to perform the
offline refresh but this was not acted upon.

 

 

On Saturday 06 April 2019,  Certificate revocation List (CRL) and the
manifest file of AFRINIC RPKI repository expired (around 07:24AM UTC).
Our monitoring system picked this up. The immediate action was to
generate new certificates and manifest file and upload them onto RPKI
engine system.

 

The failure was as a result of human error, no changes were made on the
system but we have taken additional steps to the existing process to
ensure that this does not happen again. We do acknowledge that it is
unacceptable to have such a failure with critical infrastructure and
necessary done in this regard.

 

 

We do apologize for the inconvenience caused and thank you for your
patience in this regard.

-- 
_______________________________________________________________
Cedrick Adrien Mbeyet                                           
Infrastructure Unit Manager, AFRINIC Ltd.
t:  +230 403 5100 / 403 5115 | f: +230 466 6758 | tt: @afrinic | w: www.afrinic.net
facebook.com/afrinic | flickr.com/afrinic | youtube.com/afrinicmedia
______________________________________________________

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.afrinic.net/pipermail/community-discuss/attachments/20190408/3c3fdedc/attachment.html>