[Community-Discuss] [rpd] Cloud Innovation Displays Very Poor, If Not Criminal, Netizenship

[Arnaud AMELINA]

Hello, community

+1 @Gregoire and @Mark Tinka

*cloud innovation*  were allocated  *big bunch of IPv4* space as a *LIR*
with *no ASN*. Interesting, and no *v6*

While the bylaws defines LIR as followed:
++++++
Local Internet Registry (LIR):
any Network Operator that provides Internet services to distinct end-users
and end-sites
++++++

I wonder  which network does cloud innovation operate and which internet
services it provides to end-users and end-sites in *Africa*.

How does this network *managing 3 x /11 of IPv4* *operate*?

There is something here for the community to learn about.

Regards

--
Arnaud

Le sam. 23 mai 2020 à 14:20, Gregoire EHOUMI via RPD <rpd at afrinic.net> a
écrit :


> Hello,

>

> Thanks Mark for exposing the details of the SEACOM AS37353 hijacking.

>

> I carefully read your report and also the Cloud Innovation Limited quick

> response including their attachments as justifications.

>

> I note that;

>

> ⁃ the service contract with Cloud Innovation covering the announcement of

> their prefixes by SEACOM AS37353 was terminated  by SEACOM.

>

> ⁃ some stale IRR route objects existed after termination of the contract.

>

> ⁃ through some multiple layer distribution an organisation in Manila

> Philippines was “delegated“ an IP block from Cloud Innovation address space.

>

> ⁃ both upstream ISP and the customer in Manila set up a BGP session using

> SEACOM's AS37353 as the ASN of the Manila customer.

>

> ⁃ there was a prompt reaction from the involved parties that included

> apologies to SEACOM and the wider internet community.

>

> ⁃ there were promises from said parties to be a better netizen which would

> mean, them not hijacking other networks ASN's.

>

> ⁃ there was clear refusal to disclose the details of the customer in

> Manila Philippines who hijacked the affected SEACOM ASN.

>

> All put together, demonstrates that what happened was an impersonation and

> not a BGP configuration error, nor an oversight in checking the right to

> use of the SEACOM ASN.

>

> 1. Why is it that the real customer did not bother presenting its

> apologies to the community

>

> 2. Why is there refusal to reveal customer’s details?

>

> 3. Why is it that the said prefix is no longer seen in the routing table

> originated by the genius ASN or any other ASN?

>

> 4. Which networks were involved and what happened to the end users?

>

> Can someone from AFRINIC explain what “delegation of IP block” mean?

>

> Note: The self organised Internet knows how to deal with bad net citizens.!

>

> Best regards

> Gregoire Ehoumi

>

>

> -------- Original message --------

> From: Lu Heng <h.lu at anytimechinese.com>

> Date: 2020-05-09 5:43 a.m. (GMT-05:00)

> To: Mark Tinka <mark.tinka at seacom.mu>

> Cc: "rpd at afrinic.net >> AfriNIC Resource Policy Discussion List" <

> rpd at afrinic.net>

> Subject: Re: [rpd] Cloud Innovation Displays Very Poor, If Not Criminal,

> Netizenship

>

> To whom it may concern,

>

> On May 8, Mark Think posted a claim to multiple lists that Cloud

> Innovation was abusing an ASN (37353) that didn’t belong to them (Cloud

> Innovation) but rather belonged to Seacom through their acquisition of

> MacroLAN.

>

> While we regret this unfortunate incident, Mark’s claims that it was

> criminal or bad netizenship on the part of Cloud Innovation is without

> foundation and utterly incorrect.

>

> As shown below in the attached document from Paul Wollner(Ex-CTO of

> Macrolan who created IRR routes to allow Macrolan to announce Cloud

> Innovation's prefix); letter from Link Infinity International Ltd. (Link

> Infinity), A customer of Cloud Innovation; and attached LOA from LARUS

> authorizing IPDC Solutions to announce the prefix with origin AS134190.

> And a Letter from IPDC. This was an innocent mistake committed by third

> parties and had nothing to do with any action by Cloud Innovation or LARUS.

>

> Here’s what happened:

>

> Cloud Innovation delegated a /24 to Link Infinity, an ISP in December 2019.

>

>

> Link Infinity further delegated that same /24 to IPDC and asked Cloud

> innovation to issue an LOA, which we did. The LOA specifically required

> IPDC to use its own ASN to announce the space (AS134190).

>

> IPDC subsequently authorized one of its customers to use the said prefix.

>

>

> For reasons still unknown to Cloud Innovation, IPDC and their customer set

> up a BGP session wherein their customer used AS37353 as the origin to

> advertise prefix 156.241.3.0/24.

>

>

> Upon discovering the announcement, rather than contact Cloud Innovation,

> Mark contacted IPDC who provided him with an incomplete explanation blaming

> their customer and Mark, not realizing that Cloud Innovation was not the

> customer in question posted far and wide about the event. It is unclear to

> us why he chose to do this rather than contact us to try and resolve the

> issue.

>

>

> A contributing factor to the erroneous BGP configuration by IPDC's

> customer may have been data contained in some outdated IRR route objects

> for 156.241.0.0/16 which have subsequently been deleted.

>

> As soon as we became aware of the problem (via Mark’s email), we began to

> investigate the situation. As soon as it was clear that this was the result

> of third-party actions, we reached out to Mark privately to let him know

> what we knew and that we were still investigating. We delayed making a

> public statement in order to try and ascertain all of the facts of the

> situation. We prefer not to make public statements based on incomplete

> information.

>

> We apologize to the community for our small part in this unfortunate

> incident and assure you that we work very hard to remain good netizens and

> will address any concerns promptly when they come to our attention.

>

>

> Sincerely,

>

> Lu Heng

> CEO

> Cloud Innovations

>

> Attached:

> 1. Letter from Paul Wollner

> 2. Letter from Link Infinity

> 3. LOA Issued to IPDC Solutions for announcing 156.241.3.0/24 from

> AS134190

>         4.     Letter from IPDC

>

> FYI: LARUS is proving IP management service for Cloud Innovation, while

> LARUS is also providing IP management service to other third parties in all

> regions, for full disclosure, LARUS and Cloud Innovation are headed by same

> CEO.

>

> Content of those letters have been posted here for your convince:

>

> *IPDC:*

>

> FOR IMMEDIATE RELEASE [Perusal of Cloud Innovation Ltd]

>

>

> IPDC Solutions Pte Ltd (IPDC) is a customer of Cloud Innovation Ltd and

> over the course of our corporate relationship we were given the authority

> to use address block 156.241.3.0/24 since 9th December 2019.

>

>

> On 12th December 2019, we have delegated that address block to our client.

> Following which our client further instructed us to announce the prefix

> with AS37353. In good will after minimal verification through WHOIS’ IP

> Prefix we have proceeded to execute our client’s request.

>

>

> On 7th May 2020 IPDC was then contacted by SEACOM, the legitimate holder

> of record for that ASN and have since learned that the customer’s use of

> that ASN was erroneous and not permitted by SEACOM and immediate action has

> been taken to rectify this situation.

>

>

> IPDC would like to apologize for our lack of attention in conducting

> thorough verification and wish to highlight that the involvement of Cloud

> Innovation Ltd in the entire process was providing the addresses to us and

> that we truly apologize as we understand that this incident may have

> indirectly implicated Cloud Innovation Ltd.

>

>

> IPDC however, does not wish to disclose our client information and our

> client information shall remain confidential under present circumstances.

> Once again, we apologize for our shortcomings.

>

>

> *Link Infinity:*

>

>

> To whom it may concern,

>

> We at HK Infinity International Ltd are a customer of Cloud Innovation and

> in the course received rights to use 156.241.3.0/24 from them. Beginning

> December, 2019, we delegated the right to announce this prefix to IPDC

> Solutions Pte Ltd. (IPDC). We asked Cloud Innovation to provide an LOA

> authorizing them to announce the space which was subsequently received.

> IPDC subsequently and without our knowledge delegated this space to one of

> their customers and allowed them to originate it from AS37353.

>

> This announcement was not authorized by us, nor is it permitted by the LOA

> provided by Cloud Innovation.

>

> Unfortunately, we were not aware of the situation until after it had

> already been resolved.

>

> It was never our intent to violate the LOA or to allow the prefix to be

> announced from a hijacked ASN. We do not condone this and apologize

> sincerely to the community for what has happened here.

>

> Sincere Apologies,

>

>

> *Paul Wollner:*

>

>

> 8 May 2020

>

> TO WHOM IT MAY CONCERN

>

> In the light of the recent email on NAPAfrica mailing list, I would just

> like to clear the air.

>

>

> The IP route objects were created by myself for Cloud Innovation when they

> signed up as a client of Macrolan ( now SEACOM) as they didn't have their

> own AS.

>

> At the time I was working for Macrolan (now SEACOM). I left the employment

> of SEACOM in October 2019.

>

> It appears that when Cloud Innovation's contract with SEACOM came to an

> end, the route objects were never cleaned up.

>

> This occurred after I left SEACOM's employment. Since leaving, I have no

> access to these objects.

>

> Regards

>

> Paul Wollner

> 082-786-9776

>

>

>

> On Fri, 8 May 2020 at 22:10, Mark Tinka <mark.tinka at seacom.mu> wrote:

>

>> Hi all.

>>

>> I'm not one to b**ch & moan in public, but per subject, I could not let

>> this one slide.

>>

>> And if you get this from multiple mailing lists, apologies for that - I'm

>> just trying to make sure that this reaches as wide an audience as possible,

>> on the continent.

>>

>> SEACOM (AS37100) acquired MacroLan (AS37353) a couple of years ago.

>> MacroLan is now part of the SEACOM family, and while we are in the process

>> of integrating that network into AS37100, some existing services continue

>> to be delivered on AS37353 until that exercise is completed.

>>

>> One of the customers that AS37353 was providing services to was Cloud

>> Innovation, in Cape Town. From a routing perspective, because Cloud

>> Innovation had no AS number for this service, all of their IP address space

>> was being originated by AS37353, on their behalf.

>>

>> In December of 2019, AS37353 ceased providing services to Cloud

>> Innovation. That is 6 months ago.

>>

>> In recent days, it came to SEACOM's attention that some of Cloud

>> Innovation's IP address space was being originated by AS37353 -

>> specifically, 156.241.3.0/24 - even though we were sure that they were

>> no longer a customer of AS37353 since December of 2019. At first, we

>> thought this was a cached entry in a number of popular looking glasses, but

>> then every looking glass had the same entry, which made this an unlikely

>> bug.

>>

>> As of yesterday afternoon, see below what Telia's looking glass was

>> saying about this prefix:

>>

>> *****

>>

>> Path #1: Received by speaker 0

>>   4809 134190 37353

>>     2.255.249.42 (metric 2134) from 2.255.253.101 (80.91.242.40)

>>       Origin incomplete, localpref 200, valid, internal, best,

>> group-best, import-candidate

>> Communities:

>>

>> 1299:431

>>     (RPKI state Unknown)

>>

>> 1299:1000 1299:30000 1299:30600 23456:20413 45352:4500 45352:9204

>>

>> *****

>>

>> But when I run a traceroute from my house to that prefix, it clearly was

>> not ending up in Cape Town, where AS37353's main operation resides:

>>

>> *****

>>

>> MacBook-Pro-7:~ tinka$ traceroute -I 156.241.3.1

>> traceroute to 156.241.3.1 (156.241.3.1), 64 hops max, 72 byte packets

>>  1  172.16.0.254 (172.16.0.254)  14.824 ms  11.522 ms  3.525 ms

>>  2  xe-1-3-0-1064.er-01-jnb.za.seacomnet.com (105.22.37.13)  5.620 ms

>> 9.714 ms  9.887 ms

>>  3  ce-0-2-0-0.cr-02-jnb.za.seacomnet.com (105.16.28.2)  175.232 ms

>> 172.699 ms  175.896 ms

>>  4  xe-0-0-0-8.cr-02-cpt.za.seacomnet.com (105.16.9.182)  164.496 ms

>> 163.578 ms  163.546 ms

>>  5  105.16.14.153 (105.16.14.153)  169.812 ms  171.272 ms  177.115 ms

>>  6  xe-0-0-0-0.br-02-lhr.uk.seacomnet.com (105.16.34.253)  168.911 ms

>> 172.958 ms  165.165 ms

>>  7  82.112.115.169 (82.112.115.169)  172.700 ms  176.482 ms  174.375 ms

>>  8  ae-17.r05.londen12.uk.bb.gin.ntt.net (129.250.2.147)  672.099 ms

>> 613.617 ms  615.109 ms

>>  9  ae-2.r24.londen12.uk.bb.gin.ntt.net (129.250.4.244)  181.952 ms

>> 183.087 ms  187.302 ms

>> 10  ae-16.r20.frnkge13.de.bb.gin.ntt.net (129.250.3.13)  190.511 ms

>> 185.579 ms  187.058 ms

>> 11  ae-3.r20.sngpsi07.sg.bb.gin.ntt.net (129.250.4.17)  520.882 ms

>> 613.982 ms  615.275 ms

>> 12  ae-9.r24.tkokhk01.hk.bb.gin.ntt.net (129.250.7.67)  612.301 ms

>> 586.886 ms  436.711 ms

>> 13  ae-1.r03.tkokhk01.hk.bb.gin.ntt.net (129.250.6.98)  614.470 ms

>> 613.416 ms  614.281 ms

>> 14  ce-0-3-0-3.r03.tkokhk01.hk.ce.gin.ntt.net (203.131.241.126)  614.128

>> ms  613.661 ms  615.416 ms

>> 15  * *     *

>> 16  * * *

>> 17  156.241.3.1 (156.241.3.1)  494.400 ms  410.180 ms *

>> MacBook-Pro-7:~ tinka$

>>

>> *****

>>

>> So we, then, realized that this was a fraudulent use of MacroLan's

>> AS37353, to which we had given no express permission.

>>

>> As luck would have it, due to my days living and working in Malaysia, I

>> know the good folk that operate AS134190 (IPDC Solutions), who was the

>> upstream providing transit for this prefix. So I rang them up yesterday

>> afternoon, told them what was happening, and within the hour, they got that

>> eBGP session shutdown. I am most grateful to them for their quick response

>> and immediate understanding of what was going on. Even with all the

>> technology we have today, it, many times, comes down to having good friends

>> in good places.

>>

>> Anyway, it turns out the ISP that had acquired this prefix from Cloud

>> Innovation is based in Manila, Philippines. When IPDC delivered their

>> transit service to them in Manila, that ISP informed them that they should

>> use AS37353 for the eBGP session. Yes, one could argue that IPDC should

>> have done their checks to ensure that the AS number being provided by their

>> customer belongs to them, but to be honest, I'm not too bothered about that

>> compared to Cloud Innovation's thinking that it was okay to use another

>> network's AS number in order to deliver their services to their customers.

>>

>> IPDC confirm that this service was activated for the Manila ISP in

>> December of 2019, right around the time Cloud Innovation's service with

>> SEACOM, in Cape Town, ended.

>>

>> As it currently stands, the ISP in Manila is now off the Internet, with

>> some 12 paying customers currently without service. Their excuse - they do

>> not have an AS number of their own.

>>

>> IPDC tried to find out why the ISP in Manila thought that it was okay to

>> use another network's AS number for their service, and as it turns out, the

>> network engineer at the Manila ISP that set this up has since left the

>> company. All the ones currently there do not have any history about all of

>> this.

>>

>> Currently, 156.241.3.0/24 is not in the global BGP table:

>>

>> *****

>>

>> lg-01-ams.nl>sh ip bgp 156.241.3.0/24

>> % Network not in table

>> lg-01-ams.nl>

>>

>> lg-01-nbo.ke>sh ip bgp 156.241.3.0/24

>> % Network not in table

>> lg-01-nbo.ke>

>>

>> lg-01-cpt.za>sh ip bgp 156.241.3.0/24

>> % Network not in table

>> lg-01-cpt.za>

>>

>> *****

>>

>> That Cloud Innovation thought it was okay for them to use MacroLan's AS

>> number to originate their own prefixes into the global BGP is as morally

>> reprehensible as it is technologically distasteful.

>>

>> SEACOM have been working very closely with AFRINIC to delete previous

>> route objects from their IRR that certify a relationship between Cloud

>> Innovation's parent /16 aggregates that cover this prefix, and AS37353, but

>> this is one of those objects that cannot be removed via the MyAFRINIC

>> portal, and requires manual intervention from AFRINIC.

>>

>> I have not, personally, spoken to the proprietors of Cloud Innovation

>> and/or Outside Heaven, as I don't see how anything could explain this with

>> any degree of justification.

>>

>> For now, I will find some beer to wipe the foul taste from my mouth,

>> while we (SEACOM) consider what to do about this.

>>

>> And yes, for those who are wondering, RPKI's ROV would not have prevented

>> this, in its current form. This is AS hijacking, and ROV, today, tries to

>> solve the prefix-hijacking problem, first.

>>

>> Thank you for your attention.

>>

>> Mark.

>> _______________________________________________

>> RPD mailing list

>> RPD at afrinic.net

>> https://lists.afrinic.net/mailman/listinfo/rpd

>>

>

>

> --

> --

> Kind regards.

> Lu

>

> _______________________________________________

> RPD mailing list

> RPD at afrinic.net

> https://lists.afrinic.net/mailman/listinfo/rpd

>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.afrinic.net/pipermail/community-discuss/attachments/20200525/9e76e35e/attachment.html>