<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p>
</p>
<p class="MsoNormal"><span style="mso-ansi-language:EN-US">Dear
AFRINIC community,</span></p>
<p class="MsoNormal"><span style="mso-ansi-language:EN-US"><br>
</span></p>
<p class="MsoNormal"><span style="mso-ansi-language:EN-US">Find
below postmortem report on the incident that happen on 06 April
2019. <br>
</span></p>
<p class="MsoNormal"><span style="mso-ansi-language:EN-US"> </span></p>
<p class="MsoNormal"><span style="mso-ansi-language:EN-US">The
AFRINIC RPKI
engine has an offline part that has to be renewed on a monthly
bases. The
process is known, documented and automated reminders set. The
system is set to
send 2 reminders each month, one 15 days prior to the expiry
date and the
second one 7 days before expiry. On the 2nd half of March, the
monitoring
system sent a reminder to perform the offline refresh but this
was not acted
upon. </span></p>
<p class="MsoNormal"><span style="mso-ansi-language:EN-US"> </span></p>
<p class="MsoNormal"><span style="mso-ansi-language:EN-US"> </span></p>
<p class="MsoNormal"><span style="mso-ansi-language:EN-US">On
Saturday 06 April
2019,<span style="mso-spacerun:yes"> </span>Certificate
revocation List (CRL)
and the manifest file of AFRINIC RPKI repository expired (around
07:24AM UTC).
Our monitoring system picked this up. The immediate action was
to generate new
certificates and manifest file and upload them onto RPKI engine
system.</span></p>
<p class="MsoNormal"><span style="mso-ansi-language:EN-US"> </span></p>
<p class="MsoNormal"><span style="mso-ansi-language:EN-US">The
failure was as a
result of human error, no changes were made on the system but we
have taken
additional steps to the existing process to ensure that this
does not happen
again. We do acknowledge that it is unacceptable to have such a
failure with
critical infrastructure and necessary done in this regard. </span></p>
<p class="MsoNormal"><span style="mso-ansi-language:EN-US"> </span></p>
<p class="MsoNormal"><span style="mso-ansi-language:EN-US"> </span></p>
<span
style="font-size:12.0pt;font-family:"Calibri",sans-serif;mso-ascii-theme-font:minor-latin;mso-fareast-font-family:Calibri;mso-fareast-theme-font:minor-latin;
mso-hansi-theme-font:minor-latin;mso-bidi-font-family:"Times
New Roman";
mso-bidi-theme-font:minor-bidi;mso-ansi-language:EN-US;mso-fareast-language:
EN-US;mso-bidi-language:AR-SA">We do apologize for the
inconvenience caused and
thank you for your patience in this regard.</span>
<p>
<style>
<!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;
mso-font-charset:0;
mso-generic-font-family:roman;
mso-font-pitch:variable;
mso-font-signature:-536870145 1107305727 0 0 415 0;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;
mso-font-charset:0;
mso-generic-font-family:swiss;
mso-font-pitch:variable;
mso-font-signature:-536859905 -1073732485 9 0 511 0;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-parent:"";
margin:0cm;
margin-bottom:.0001pt;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-ansi-language:EN-GB;}
.MsoChpDefault
{mso-style-type:export-only;
mso-default-props:yes;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-ansi-language:EN-GB;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;
mso-header-margin:36.0pt;
mso-footer-margin:36.0pt;
mso-paper-source:0;}
div.WordSection1
{page:WordSection1;}
-->
</style></p>
<pre class="moz-signature" cols="72">
--
_______________________________________________________________
Cedrick Adrien Mbeyet
Infrastructure Unit Manager, AFRINIC Ltd.
t: +230 403 5100 / 403 5115 | f: +230 466 6758 | tt: @afrinic | w: <a class="moz-txt-link-abbreviated" href="http://www.afrinic.net">www.afrinic.net</a>
facebook.com/afrinic | flickr.com/afrinic | youtube.com/afrinicmedia
______________________________________________________
</pre>
</body>
</html>